-- Logs begin at Fri 2022-10-14 02:17:38 UTC, end at Fri 2022-10-14 03:33:44 UTC. -- Oct 14 02:48:02.848167 dv-cinder-801522-13-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Oct 14 02:48:02.857156 dv-cinder-801522-13-iscsi devstack@keystone.service[81507]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Oct 14 02:48:02.859196 dv-cinder-801522-13-iscsi devstack@keystone.service[81507]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Oct 14 02:48:02 2022] *** Oct 14 02:48:02.859196 dv-cinder-801522-13-iscsi devstack@keystone.service[81507]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 14 02:48:02.859196 dv-cinder-801522-13-iscsi devstack@keystone.service[81507]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 14 02:48:02.859196 dv-cinder-801522-13-iscsi devstack@keystone.service[81507]: nodename: dv-cinder-801522-13-iscsi Oct 14 02:48:02.859196 dv-cinder-801522-13-iscsi devstack@keystone.service[81507]: machine: x86_64 Oct 14 02:48:02.859196 dv-cinder-801522-13-iscsi devstack@keystone.service[81507]: clock source: unix Oct 14 02:48:02.859196 dv-cinder-801522-13-iscsi devstack@keystone.service[81507]: pcre jit disabled Oct 14 02:48:02.859196 dv-cinder-801522-13-iscsi devstack@keystone.service[81507]: detected number of CPU cores: 4 Oct 14 02:48:02.859196 dv-cinder-801522-13-iscsi devstack@keystone.service[81507]: current working directory: / Oct 14 02:48:02.859196 dv-cinder-801522-13-iscsi devstack@keystone.service[81507]: detected binary path: /usr/bin/uwsgi-core Oct 14 02:48:02.859196 dv-cinder-801522-13-iscsi devstack@keystone.service[81507]: your processes number limit is 47680 Oct 14 02:48:02.859787 dv-cinder-801522-13-iscsi devstack@keystone.service[81507]: your memory page size is 4096 bytes Oct 14 02:48:02.859787 dv-cinder-801522-13-iscsi devstack@keystone.service[81507]: detected max file descriptor number: 2048 Oct 14 02:48:02.859787 dv-cinder-801522-13-iscsi devstack@keystone.service[81507]: lock engine: pthread robust mutexes Oct 14 02:48:02.859787 dv-cinder-801522-13-iscsi devstack@keystone.service[81507]: thunder lock: enabled Oct 14 02:48:02.859787 dv-cinder-801522-13-iscsi devstack@keystone.service[81507]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Oct 14 02:48:02.859787 dv-cinder-801522-13-iscsi devstack@keystone.service[81507]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 14 02:48:02.898547 dv-cinder-801522-13-iscsi devstack@keystone.service[81507]: Python main interpreter initialized at 0x563cdb24c9c0 Oct 14 02:48:02.898547 dv-cinder-801522-13-iscsi devstack@keystone.service[81507]: python threads support enabled Oct 14 02:48:02.898547 dv-cinder-801522-13-iscsi devstack@keystone.service[81507]: your server socket listen backlog is limited to 100 connections Oct 14 02:48:02.898547 dv-cinder-801522-13-iscsi devstack@keystone.service[81507]: your mercy for graceful operations on workers is 90 seconds Oct 14 02:48:02.898860 dv-cinder-801522-13-iscsi devstack@keystone.service[81507]: mapped 403077 bytes (393 KB) for 2 cores Oct 14 02:48:02.898956 dv-cinder-801522-13-iscsi devstack@keystone.service[81507]: *** Operational MODE: preforking *** Oct 14 02:48:02.899222 dv-cinder-801522-13-iscsi devstack@keystone.service[81507]: *** uWSGI is running in multiple interpreter mode *** Oct 14 02:48:02.899222 dv-cinder-801522-13-iscsi devstack@keystone.service[81507]: spawned uWSGI master process (pid: 81507) Oct 14 02:48:02.899405 dv-cinder-801522-13-iscsi systemd[1]: Started Devstack devstack@keystone.service. Oct 14 02:48:02.900183 dv-cinder-801522-13-iscsi devstack@keystone.service[81507]: spawned uWSGI worker 1 (pid: 81517, cores: 1) Oct 14 02:48:02.900287 dv-cinder-801522-13-iscsi devstack@keystone.service[81507]: spawned uWSGI worker 2 (pid: 81518, cores: 1) Oct 14 02:48:02.900807 dv-cinder-801522-13-iscsi devstack@keystone.service[81507]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 14 02:48:07.236391 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 14 02:48:07.236391 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: from cryptography.utils import int_from_bytes Oct 14 02:48:07.237182 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 14 02:48:07.237182 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: from cryptography.utils import int_from_bytes Oct 14 02:48:07.241121 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 14 02:48:07.241121 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: from cryptography.utils import int_from_bytes Oct 14 02:48:07.241583 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 14 02:48:07.241583 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: from cryptography.utils import int_from_bytes Oct 14 02:48:07.287203 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 14 02:48:07.287342 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 14 02:48:07.287512 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] command line args: [] {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 14 02:48:07.287810 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 14 02:48:07.287988 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] ================================================================================ {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 14 02:48:07.288246 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] admin_token = **** {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.288492 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] config_dir = [] {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.288680 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.288862 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] config_source = [] {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.289045 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] debug = True {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.289240 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.290175 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.290175 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.290175 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.290397 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.290567 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] list_limit = None {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.290714 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] log_config_append = None {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.290856 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.291034 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] log_dir = None {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.291210 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] log_file = None {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.291455 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.291690 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.291935 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.292181 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.292357 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.292532 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.292776 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.293019 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.293273 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.293536 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.293736 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.293992 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.294238 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.294481 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.294763 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.295246 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.39/identity {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.295430 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] publish_errors = False {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.295661 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.295888 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.296066 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.296311 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.296546 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.296781 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] standard_threads = False {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.297011 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.297174 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 14 02:48:07.297264 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 14 02:48:07.297342 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.297402 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] command line args: [] {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 14 02:48:07.297482 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.297560 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] use_journal = False {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.297636 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 14 02:48:07.297739 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] use_json = False {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.297838 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] ================================================================================ {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 14 02:48:07.297938 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] use_stderr = False {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.303079 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] admin_token = **** {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.303173 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] use_syslog = False {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.303247 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] config_dir = [] {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.303322 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.303398 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.303494 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.303574 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] config_source = [] {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.303651 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.303721 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] debug = True {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.303721 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.303879 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.303962 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.304034 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.304109 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.304180 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.304248 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.304317 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.304393 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.304472 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.304548 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] list_limit = None {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.304628 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.304703 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] log_config_append = None {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.304703 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.304843 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.304952 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] log_dir = None {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.305024 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.305100 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] log_file = None {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.305241 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.305323 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.305394 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.305463 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.305537 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.305612 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.306214 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.306295 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.306295 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.306424 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.306493 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.306563 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.306640 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.306718 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.306794 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.306872 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.306949 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.307031 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.307103 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.307172 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.307241 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.307315 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.307392 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.307474 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.307551 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.307551 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.307689 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.307689 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.307821 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.307971 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.307971 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.308109 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.39/identity {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.308109 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] publish_errors = False {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.308242 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.308314 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.308382 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.308457 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.308533 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.308609 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.308687 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.308759 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.308829 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.308983 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.309060 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.309130 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] standard_threads = False {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.309209 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.309280 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.309349 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.309417 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.309486 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.309555 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.309555 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] use_journal = False {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.309688 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.309778 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] use_json = False {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.309862 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.309942 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] use_stderr = False {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.309942 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] use_syslog = False {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.310090 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.310162 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 14 02:48:07.310237 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.310237 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.310511 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.310621 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.310713 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.310785 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.310861 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.310932 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.311006 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.311076 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.311154 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.311230 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.311300 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.311375 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.311445 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.311514 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.311654 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.311733 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.311804 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.311878 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.311948 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.312023 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.312104 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.312176 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.312245 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.312320 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.312389 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.312464 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.312541 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.312611 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.312680 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.312748 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.312816 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.312890 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.312968 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.313038 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.313112 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.313216 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.313313 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.313426 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.313528 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.313623 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.313718 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.313838 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.313838 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.314043 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.314142 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.314238 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.314343 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.314422 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.314570 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.314655 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.314779 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.314856 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.314978 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.315054 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.315126 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.315252 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.315328 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.315400 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.315515 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.315590 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.315712 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.315800 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.315884 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.315962 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.316041 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.316119 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.316236 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.316311 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.316433 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.316518 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.316590 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.316705 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.316786 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.316858 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.316979 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.317063 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.317146 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.317219 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.317333 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.317409 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.317524 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.317607 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.317680 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.317809 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.317925 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.318001 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.318146 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.318280 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.318384 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.318465 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.318619 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.318709 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.318852 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.318939 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.319092 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.319178 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.319326 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.319405 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.319563 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.319649 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.319795 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.319873 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.320033 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.320118 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.320263 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.320348 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.320496 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.320582 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.320731 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.320809 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.320958 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.321037 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.321186 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.321265 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.321476 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.322138 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.322300 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.322385 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.322463 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.322535 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.322673 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.322753 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.322909 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.322997 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.323148 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.323227 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.323385 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.323464 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.323635 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.323718 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.323869 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.323949 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.324109 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.324186 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.324337 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] auth.external = None {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.324415 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.324567 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.324649 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.324832 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.324913 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.325079 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.325162 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.325315 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] auth.password = None {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.325394 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.325545 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] auth.token = None {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.325623 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.325795 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.325888 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.326038 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.326117 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.326283 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.326360 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.326514 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.326592 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.326750 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.326834 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.326995 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.327073 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.327222 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.327299 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.327458 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] credential.caching = True {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.327537 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.327694 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.327772 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.327931 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.328009 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.328168 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.328246 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.328405 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.328483 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.328641 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.328726 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.328873 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.328982 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.329119 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.329200 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.329370 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.329448 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.329603 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.329688 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.329871 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.329985 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.330118 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.330214 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.330360 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.330439 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.330598 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.330705 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.330837 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.330948 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.331071 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.331204 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.331294 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.331439 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] federation.caching = True {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.331519 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.331676 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.331755 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.331922 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.332003 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.332170 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.332249 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.332401 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.332488 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.332648 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.332729 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.332883 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.332962 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.333135 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.333216 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.333370 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.333536 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.333615 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.333674 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.333872 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] auth.external = None {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.333954 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.334108 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.334189 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.334367 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] identity.caching = True {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.334444 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.334613 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.334691 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.334846 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] auth.password = None {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.334925 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.335078 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] auth.token = None {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.335157 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.335316 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.335394 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.335538 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.335611 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.335849 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.335949 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.336030 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.336103 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.336245 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.336327 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.336493 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.336572 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.336721 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.336799 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.336981 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] credential.caching = True {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.337066 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.337215 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.337294 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.337459 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.337538 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.337707 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.337813 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.337979 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.338057 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.343211 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.343353 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.343441 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.343515 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.343586 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.343658 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.343734 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.343806 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.343884 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.343956 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.344027 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.344087 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.344087 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.344214 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.344292 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.344370 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.344441 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.344512 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.344677 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.344753 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.344823 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.344895 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.344965 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.345036 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] federation.caching = True {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.345105 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.345175 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.345254 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.345325 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.345396 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.345473 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.345544 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.345622 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.345693 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.345805 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.345880 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.345952 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.346022 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.346104 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.346178 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.346255 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.346326 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.346396 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.346466 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.346807 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.346892 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.346964 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.347034 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.347104 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] identity.caching = True {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.347174 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.347253 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.347325 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.347396 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.347396 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.347524 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.347594 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.347672 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.347744 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.347813 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.347883 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.347953 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.348022 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.348099 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.348171 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.348241 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.348311 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.348380 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.348450 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.348531 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.348603 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.348673 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.348750 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.348822 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] ldap.user = None {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.348881 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.348947 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.349048 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.349121 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.349193 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.349263 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.349333 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.349411 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.349483 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.349553 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.349623 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.349693 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.349789 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.349876 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.349950 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.350021 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.350091 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.350160 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.350230 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.350313 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.350386 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.350456 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.350530 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.350601 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.350751 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.350894 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.350970 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.351041 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.351112 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.351182 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.351252 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.351330 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.351403 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.351473 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.351543 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.351613 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.351683 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.351761 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.351833 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.351902 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.351973 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.352043 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.352113 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.352191 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.352263 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.352333 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.352403 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.352472 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.352542 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.352620 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.352692 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.352761 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.352827 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.352885 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.352944 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.353009 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.353068 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.353157 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.353239 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.353394 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.353474 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.353630 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.353711 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.353890 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.353970 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.354140 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.354231 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.354383 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.354462 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.354886 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.354964 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.355122 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.355209 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.355367 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] resource.caching = True {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.355446 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.355616 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.355694 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] ldap.user = None {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.355862 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.355949 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.356098 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.356176 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.356345 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.356424 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.356592 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.356679 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.356827 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.356906 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.357072 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.357223 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.357299 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.357373 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.357535 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.357615 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.357805 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.357889 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] role.caching = True {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.358036 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.358194 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] role.driver = sql {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.358270 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.358341 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.358496 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.358576 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.358748 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.358835 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.358998 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.359076 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.359245 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.359323 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.359491 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.359577 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.359735 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.359813 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.359965 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.360043 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.360212 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.360298 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.360441 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.360519 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.360676 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.360754 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.360913 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.361000 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.361151 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.361230 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.361394 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.361472 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.361640 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.361946 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.362043 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.362167 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.362247 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.362400 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.362480 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.362647 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.362728 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.362885 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.362964 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.363127 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.363206 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.363359 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.363439 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.363606 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.363683 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.363850 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.363937 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.364086 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.364164 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.364315 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.364393 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.364548 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.364636 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] resource.caching = True {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.364780 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.364860 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.365033 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.365111 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.365271 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.365359 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.365522 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.365600 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.365781 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.365862 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.366021 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.366111 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.366253 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.366333 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.366490 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] token.caching = True {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.366569 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.366730 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.366817 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.366974 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] role.caching = True {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.367085 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.367206 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.367290 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] role.driver = sql {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.367444 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.367541 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.367693 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.367775 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.367926 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.368006 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.368678 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.368780 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.368924 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.369013 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.369173 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.369262 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.369411 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.369510 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.369651 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.369740 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.369911 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.370016 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.370144 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.370250 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.370394 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.370481 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.370636 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.370724 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.370879 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.370960 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.371126 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.371205 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.371393 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] database.connection = **** {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.371520 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.371663 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.371749 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.371884 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.371999 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.372122 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.372251 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.372363 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.372502 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.372590 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.372753 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.372832 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.372990 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.373069 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.373247 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.373335 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.373498 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.373577 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.373726 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.373930 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.374007 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.374172 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.374250 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.374310 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.374424 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.374544 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.374677 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.374765 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.374933 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.375012 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.375182 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.375261 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.375421 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.375508 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.375659 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.375738 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.375837 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81518) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 14 02:48:07.375916 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.376117 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] token.caching = True {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.376355 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.376604 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.376842 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.377098 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.377348 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.377587 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.377859 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.378105 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.378354 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.378593 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.378834 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.379071 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.379365 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.379595 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.379830 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.380075 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.380346 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.380609 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] database.connection = **** {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.380854 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.381103 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.381341 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.381581 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.381838 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.382078 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.382314 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.382552 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.382798 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.383036 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.383281 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.383524 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.383824 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.384621 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.384871 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.385106 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.385349 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.385589 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 14 02:48:07.385803 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81517) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 14 02:48:07.405566 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81518) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:48:07.405824 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81518) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:48:07.406054 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81518) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:48:07.414453 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81518) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:48:07.421456 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81517) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:48:07.421691 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81517) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:48:07.421950 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81517) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:48:07.430238 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81517) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:48:07.442639 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81518) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:48:07.442866 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81518) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:48:07.449370 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81517) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:48:07.449599 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81517) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:48:07.501304 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81517) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:48:07.501449 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81518) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:48:07.567070 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81517) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:48:07.567070 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81517) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:48:07.567284 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81518) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:48:07.567284 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81518) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:48:07.728185 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81517) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:48:07.728455 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81517) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:48:07.728595 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81518) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:48:07.728750 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81517) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:48:07.728909 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81518) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:48:07.729037 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81517) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:48:07.729163 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81518) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:48:07.729302 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81517) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:48:07.729436 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81518) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:48:07.729844 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81517) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:48:07.729971 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81518) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:48:07.729971 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81518) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:48:07.754229 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81517) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:48:07.754408 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81518) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:48:07.754539 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81517) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:48:07.754666 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81518) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:48:07.754792 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81517) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:48:07.754915 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81518) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:48:07.755049 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81517) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:48:07.755188 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81518) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:48:07.755317 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81517) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:48:07.755446 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81518) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:48:07.755680 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81517) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:48:07.755836 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81518) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:48:07.755971 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81517) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:48:07.756100 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81518) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:48:07.767212 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81517) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:48:07.767390 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81518) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:48:07.780226 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81517) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:48:07.780459 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81517) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:48:07.780596 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81518) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:48:07.780819 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81517) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:48:07.780954 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81518) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:48:07.780954 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81518) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:48:07.781193 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81517) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:48:07.781193 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81517) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:48:07.781445 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81518) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:48:07.781570 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81517) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:48:07.781695 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81518) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:48:07.781867 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81517) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:48:07.782008 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81518) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:48:07.782008 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81518) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 14 02:48:07.798074 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.798282 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 14 02:48:07.798448 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.798626 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.798754 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 14 02:48:07.798881 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 14 02:48:07.799007 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.799132 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.799267 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 14 02:48:07.799401 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.799526 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.799650 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.799775 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.799901 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.800032 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.800157 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.809454 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.809736 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.809915 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.810043 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.810190 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.810319 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.810451 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.810728 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.817306 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81517) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 02:48:07.817775 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81518) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 02:48:07.820200 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81517) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 02:48:07.820484 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.820715 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81518) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 02:48:07.820799 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.821005 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.821089 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.821832 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.821924 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.822113 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.822194 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.822388 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.822471 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.822647 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.822800 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.822927 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.823021 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.823179 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.823261 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.823450 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.823530 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.823722 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.823995 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.846793 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.847341 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.848325 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81517) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 02:48:07.848550 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.848854 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81518) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 02:48:07.849143 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.853264 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81517) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 02:48:07.853553 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.853836 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81518) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 02:48:07.853936 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.854126 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.854409 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.860497 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81517) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 02:48:07.860783 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.861061 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81518) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 02:48:07.861353 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.866433 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81517) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 02:48:07.866736 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.867052 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.867143 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81518) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 02:48:07.867328 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.867412 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.867609 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.867691 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.867888 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.867980 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.868190 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.868272 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.868469 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.868551 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.868742 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.869016 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.884606 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.884895 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.885163 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.885453 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.888801 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81517) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 02:48:07.889436 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81518) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 02:48:07.892062 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.892363 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.892688 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.892989 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.896623 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81517) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 02:48:07.897268 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81518) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 02:48:07.900314 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81517) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 02:48:07.901008 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81518) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 02:48:07.903829 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.904134 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.904424 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.904553 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.904711 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.904860 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.904994 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.905149 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.905288 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.905439 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.905723 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.906031 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.921144 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81517) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 02:48:07.921439 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.921713 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.921999 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81518) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 02:48:07.922084 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.922309 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.922607 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.922871 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.928843 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.929779 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.930450 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.931369 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.931975 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.932267 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.932888 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.933178 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.934986 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81517) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 02:48:07.935274 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.935569 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.935859 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.936002 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81518) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 02:48:07.936262 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.936345 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.936575 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.936864 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.937264 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.948011 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81517) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 02:48:07.948302 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.948588 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.952058 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.952144 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81518) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 02:48:07.952209 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.952269 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.952342 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.952429 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.952505 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.952831 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.952831 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.952831 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.952831 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.965871 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81517) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 02:48:07.967053 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81518) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 02:48:07.968787 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81517) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 02:48:07.970047 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81518) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 02:48:07.971731 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.972936 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.973177 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.974438 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.974683 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81517) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 02:48:07.975000 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.975384 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.975974 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81518) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 02:48:07.976261 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.976671 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.982111 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.983409 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.983580 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81517) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 02:48:07.984917 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81518) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 02:48:07.986503 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.986910 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.987207 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.987488 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.987847 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.988245 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.988536 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.988819 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.995874 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81517) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 02:48:07.996178 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.996456 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.997315 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81518) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 02:48:07.997610 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:07.997921 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:08.003394 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81517) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 02:48:08.003683 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:08.003971 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:08.004264 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:08.009537 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:08.009537 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:08.010247 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81518) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 14 02:48:08.010333 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:08.010410 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:08.010503 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:08.010582 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:08.010658 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:08.010731 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:08.010804 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:08.010888 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:08.010963 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:08.011037 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:08.011109 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:08.011183 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:08.011264 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:08.011340 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:08.011414 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81517) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:08.011487 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:08.011487 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:08.011487 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:08.011487 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:08.011487 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:08.011487 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81518) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 14 02:48:08.041571 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x563cdb24c9c0 pid: 81517 (default app) Oct 14 02:48:08.043335 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x563cdb24c9c0 pid: 81518 (default app) Oct 14 02:48:08.044883 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddc502a-449c-45cc-8c62-518f80a10360 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:08.045172 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddc502a-449c-45cc-8c62-518f80a10360 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:08.045447 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddc502a-449c-45cc-8c62-518f80a10360 None None] PATH_INFO: `/v3` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:08.049605 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1/1] 10.222.0.39 () {50 vars in 791 bytes} [Fri Oct 14 02:48:08 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:48:15.600441 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3969d05-9761-445b-aa59-bd25f8182fe5 None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:15.600974 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3969d05-9761-445b-aa59-bd25f8182fe5 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:15.600974 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3969d05-9761-445b-aa59-bd25f8182fe5 None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:15.606210 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1/2] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:48:15 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 14 02:48:15.611306 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28db693c-6d74-46ab-b849-338062bc813b None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:15.611759 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28db693c-6d74-46ab-b849-338062bc813b None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:15.611855 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28db693c-6d74-46ab-b849-338062bc813b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:15.681973 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: INFO dbcounter [None req-28db693c-6d74-46ab-b849-338062bc813b None None] Registered counter for database keystone Oct 14 02:48:15.700602 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG oslo_db.sqlalchemy.engines [None req-28db693c-6d74-46ab-b849-338062bc813b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81517) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 14 02:48:15.706770 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writer thread running {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 14 02:48:15.886419 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG passlib.handlers.bcrypt [None req-28db693c-6d74-46ab-b849-338062bc813b None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81517) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 14 02:48:15.886718 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG passlib.handlers.bcrypt [None req-28db693c-6d74-46ab-b849-338062bc813b None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81517) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 14 02:48:15.943571 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-28db693c-6d74-46ab-b849-338062bc813b None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:48:15.945158 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-28db693c-6d74-46ab-b849-338062bc813b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:48:16.073773 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 2/3] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:48:15 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 463 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 14 02:48:16.514708 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3969d05-9761-445b-aa59-bd25f8182fe5 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:16.584038 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: INFO dbcounter [None req-a3969d05-9761-445b-aa59-bd25f8182fe5 None None] Registered counter for database keystone Oct 14 02:48:16.605408 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG oslo_db.sqlalchemy.engines [None req-a3969d05-9761-445b-aa59-bd25f8182fe5 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81518) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 14 02:48:16.611826 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writer thread running {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 14 02:48:16.730240 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3969d05-9761-445b-aa59-bd25f8182fe5 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:16.734520 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59deb44f-4b56-483b-afe7-abaf8eb5a464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:16.735152 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59deb44f-4b56-483b-afe7-abaf8eb5a464 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:16.735436 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59deb44f-4b56-483b-afe7-abaf8eb5a464 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:16.735709 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59deb44f-4b56-483b-afe7-abaf8eb5a464 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:16.744408 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59deb44f-4b56-483b-afe7-abaf8eb5a464 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:16.794989 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: WARNING keystone.common.rbac_enforcer.enforcer [None req-59deb44f-4b56-483b-afe7-abaf8eb5a464 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 14 02:48:16.795282 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59deb44f-4b56-483b-afe7-abaf8eb5a464 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:16.804443 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 2/4] 10.222.0.39 () {58 vars in 1324 bytes} [Fri Oct 14 02:48:16 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 14 02:48:16.810874 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28db693c-6d74-46ab-b849-338062bc813b None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:16.827213 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28db693c-6d74-46ab-b849-338062bc813b None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:16.831595 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a9651b-6869-470c-bb25-7088c9b9cae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:16.832678 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a9651b-6869-470c-bb25-7088c9b9cae7 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:16.832956 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a9651b-6869-470c-bb25-7088c9b9cae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:16.833232 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a9651b-6869-470c-bb25-7088c9b9cae7 admin admin] PATH_INFO: `/v3/projects/e86df8ec1d224299a71e492e2b3fb74d` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:16.834832 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a9651b-6869-470c-bb25-7088c9b9cae7 admin admin] RBAC: Authorizing `identity:get_project(project_id=e86df8ec1d224299a71e492e2b3fb74d)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:16.887433 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: WARNING keystone.common.rbac_enforcer.enforcer [None req-78a9651b-6869-470c-bb25-7088c9b9cae7 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 14 02:48:16.887785 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a9651b-6869-470c-bb25-7088c9b9cae7 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:16.892229 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 3/5] 10.222.0.39 () {56 vars in 1207 bytes} [Fri Oct 14 02:48:16 2022] GET /identity/v3/projects/e86df8ec1d224299a71e492e2b3fb74d => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:48:19.001245 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb753965-41e1-431d-a0de-b1451bd72b88 None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:19.001783 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb753965-41e1-431d-a0de-b1451bd72b88 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:19.001895 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb753965-41e1-431d-a0de-b1451bd72b88 None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:19.005977 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 3/6] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:48:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:48:19.012193 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90babba3-60b9-498c-a6b0-3301a4b1b8de None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:19.012566 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90babba3-60b9-498c-a6b0-3301a4b1b8de None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:19.012741 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90babba3-60b9-498c-a6b0-3301a4b1b8de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:19.033168 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-90babba3-60b9-498c-a6b0-3301a4b1b8de None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:48:19.034700 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-90babba3-60b9-498c-a6b0-3301a4b1b8de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:48:19.042498 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 4/7] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:48:19 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 14 02:48:19.307395 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb753965-41e1-431d-a0de-b1451bd72b88 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:19.319242 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb753965-41e1-431d-a0de-b1451bd72b88 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:19.324217 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc06f10-dbeb-4669-b80c-82e520c8f2c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:19.324823 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc06f10-dbeb-4669-b80c-82e520c8f2c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:19.325111 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc06f10-dbeb-4669-b80c-82e520c8f2c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:19.325399 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc06f10-dbeb-4669-b80c-82e520c8f2c6 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:19.333081 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc06f10-dbeb-4669-b80c-82e520c8f2c6 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:19.334881 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc06f10-dbeb-4669-b80c-82e520c8f2c6 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:19.343527 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 4/8] 10.222.0.39 () {58 vars in 1324 bytes} [Fri Oct 14 02:48:19 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 14 02:48:19.348683 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90babba3-60b9-498c-a6b0-3301a4b1b8de None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:19.357939 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90babba3-60b9-498c-a6b0-3301a4b1b8de None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:19.362181 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ffb9c6-bf5c-4eeb-a46b-2e6b8e6d3095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:19.362799 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ffb9c6-bf5c-4eeb-a46b-2e6b8e6d3095 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:19.363079 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ffb9c6-bf5c-4eeb-a46b-2e6b8e6d3095 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:19.363353 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ffb9c6-bf5c-4eeb-a46b-2e6b8e6d3095 admin admin] PATH_INFO: `/v3/users/316b39adbf9f4b0d9c13dc4c11a24e73` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:19.364642 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ffb9c6-bf5c-4eeb-a46b-2e6b8e6d3095 admin admin] RBAC: Authorizing `identity:get_user(user_id=316b39adbf9f4b0d9c13dc4c11a24e73)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:19.366633 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ffb9c6-bf5c-4eeb-a46b-2e6b8e6d3095 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:19.368665 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 5/9] 10.222.0.39 () {56 vars in 1198 bytes} [Fri Oct 14 02:48:19 2022] GET /identity/v3/users/316b39adbf9f4b0d9c13dc4c11a24e73 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:48:21.523135 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40816bb8-aafe-4645-af2c-19ce79d6f584 None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:21.523647 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40816bb8-aafe-4645-af2c-19ce79d6f584 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:21.524249 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40816bb8-aafe-4645-af2c-19ce79d6f584 None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:21.525776 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 5/10] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:48:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:48:21.530915 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22df4a2e-d6fe-4722-a136-52a2b4252eba None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:21.531319 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22df4a2e-d6fe-4722-a136-52a2b4252eba None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:21.531462 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22df4a2e-d6fe-4722-a136-52a2b4252eba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:21.549585 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-22df4a2e-d6fe-4722-a136-52a2b4252eba None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:48:21.550783 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-22df4a2e-d6fe-4722-a136-52a2b4252eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:48:21.581640 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae04e88d-50cd-4515-9632-7763fd316c38 None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:21.581964 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae04e88d-50cd-4515-9632-7763fd316c38 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:21.582248 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae04e88d-50cd-4515-9632-7763fd316c38 None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:21.583576 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 6/11] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:48:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 14 02:48:21.589534 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0476f227-9774-45e1-a9a4-18a170a30547 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:21.589854 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0476f227-9774-45e1-a9a4-18a170a30547 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:21.590130 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0476f227-9774-45e1-a9a4-18a170a30547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:21.630448 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 6/12] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:48:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:48:21.631333 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG passlib.handlers.bcrypt [None req-0476f227-9774-45e1-a9a4-18a170a30547 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81518) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 14 02:48:21.631719 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG passlib.handlers.bcrypt [None req-0476f227-9774-45e1-a9a4-18a170a30547 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81518) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 14 02:48:21.681109 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-0476f227-9774-45e1-a9a4-18a170a30547 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:48:21.682693 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-0476f227-9774-45e1-a9a4-18a170a30547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:48:21.781733 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 7/13] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:48:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 14 02:48:21.901293 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22df4a2e-d6fe-4722-a136-52a2b4252eba None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:21.925907 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:22.004317 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22df4a2e-d6fe-4722-a136-52a2b4252eba None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:22.060670 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9431ee-c837-455a-a491-5ae45b93a8f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:22.060670 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9431ee-c837-455a-a491-5ae45b93a8f2 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:22.060670 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9431ee-c837-455a-a491-5ae45b93a8f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:22.060670 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9431ee-c837-455a-a491-5ae45b93a8f2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:22.061155 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0476f227-9774-45e1-a9a4-18a170a30547 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:22.061245 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9431ee-c837-455a-a491-5ae45b93a8f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:48:22.061245 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-7c9431ee-c837-455a-a491-5ae45b93a8f2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:48:22.061245 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:48:22.061245 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:48:22.061245 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:48:22.061245 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:48:22.061245 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:48:22.061245 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:48:22.061245 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:48:22.061245 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:48:22.061245 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:48:22.061245 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:48:22.061245 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:48:22.061245 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:48:22.062047 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:48:22.062047 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:48:22.062047 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:48:22.062047 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:48:22.062047 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:48:22.062047 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:48:22.062047 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:48:22.062047 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:48:22.062047 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:48:22.062047 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:48:22.062047 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:48:22.062047 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:48:22.062047 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:48:22.062047 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:48:22.062047 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:48:22.062047 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:48:22.062047 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:48:22.062951 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:48:22.062951 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:48:22.062951 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:48:22.062951 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:48:22.062951 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:48:22.062951 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:48:22.062951 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:48:22.062951 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:48:22.062951 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:48:22.062951 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:48:22.062951 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:48:22.062951 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:48:22.062951 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:48:22.062951 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:48:22.065496 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 7/14] 10.222.0.39 () {56 vars in 1096 bytes} [Fri Oct 14 02:48:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 02:48:22.066313 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9431ee-c837-455a-a491-5ae45b93a8f2 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:22.145523 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:22.147447 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0476f227-9774-45e1-a9a4-18a170a30547 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:22.150555 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:22.163524 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9431ee-c837-455a-a491-5ae45b93a8f2 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:22.189908 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d3e7e7-6cb9-41bb-ab07-a61656c76968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:22.190774 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d3e7e7-6cb9-41bb-ab07-a61656c76968 None admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:22.191057 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d3e7e7-6cb9-41bb-ab07-a61656c76968 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:22.191337 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d3e7e7-6cb9-41bb-ab07-a61656c76968 None admin] PATH_INFO: `/v3/domains` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:22.192310 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d3e7e7-6cb9-41bb-ab07-a61656c76968 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:22.193728 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d3e7e7-6cb9-41bb-ab07-a61656c76968 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:22.197369 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fc4c55-9255-4b7a-b1a3-887ea2d0573d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:22.198004 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fc4c55-9255-4b7a-b1a3-887ea2d0573d None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:22.198283 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fc4c55-9255-4b7a-b1a3-887ea2d0573d None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:22.198549 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fc4c55-9255-4b7a-b1a3-887ea2d0573d None admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:22.199206 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fc4c55-9255-4b7a-b1a3-887ea2d0573d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:48:22.199508 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fc4c55-9255-4b7a-b1a3-887ea2d0573d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:22.200880 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fc4c55-9255-4b7a-b1a3-887ea2d0573d None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:22.213313 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 8/15] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Oct 14 02:48:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:48:22.219100 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fc4c55-9255-4b7a-b1a3-887ea2d0573d None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:22.248419 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.sql.core [None req-24d3e7e7-6cb9-41bb-ab07-a61656c76968 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 14 02:48:22.248419 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 14 02:48:22.248419 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [parameters: {'id': '3b3e7d2a6a9b427eb4d1b19475bf8aa8', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 14 02:48:22.248419 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81518) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 14 02:48:22.267937 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-24d3e7e7-6cb9-41bb-ab07-a61656c76968 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 14 02:48:22.267937 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:48:22.267937 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 14 02:48:22.267937 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 14 02:48:22.267937 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 14 02:48:22.267937 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 14 02:48:22.267937 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 14 02:48:22.267937 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application result = self._query(query) Oct 14 02:48:22.267937 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 14 02:48:22.267937 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application conn.query(q) Oct 14 02:48:22.267937 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 14 02:48:22.267937 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 14 02:48:22.267937 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 14 02:48:22.267937 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application result.read() Oct 14 02:48:22.268823 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 14 02:48:22.268823 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 14 02:48:22.268823 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 14 02:48:22.268823 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 14 02:48:22.268823 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 14 02:48:22.268823 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 14 02:48:22.268823 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 14 02:48:22.268823 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 14 02:48:22.268823 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 14 02:48:22.268823 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:48:22.268823 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 14 02:48:22.268823 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:48:22.268823 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:48:22.268823 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 14 02:48:22.268823 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 14 02:48:22.268823 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 14 02:48:22.268823 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 14 02:48:22.268823 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 14 02:48:22.269966 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application next(self.gen) Oct 14 02:48:22.269966 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 14 02:48:22.269966 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application yield resource Oct 14 02:48:22.269966 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 14 02:48:22.269966 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application next(self.gen) Oct 14 02:48:22.269966 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 14 02:48:22.269966 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application self.session.rollback() Oct 14 02:48:22.269966 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 14 02:48:22.269966 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application self.force_reraise() Oct 14 02:48:22.269966 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 14 02:48:22.269966 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise self.value Oct 14 02:48:22.269966 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 14 02:48:22.269966 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 14 02:48:22.269966 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 14 02:48:22.269966 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application session.commit() Oct 14 02:48:22.269966 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 14 02:48:22.269966 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 14 02:48:22.271153 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 14 02:48:22.271153 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application self._prepare_impl() Oct 14 02:48:22.271153 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 14 02:48:22.271153 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application self.session.flush() Oct 14 02:48:22.271153 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 14 02:48:22.271153 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application self._flush(objects) Oct 14 02:48:22.271153 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 14 02:48:22.271153 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 14 02:48:22.271153 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 14 02:48:22.271153 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application compat.raise_( Oct 14 02:48:22.271153 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 14 02:48:22.271153 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception Oct 14 02:48:22.271153 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 14 02:48:22.271153 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application flush_context.execute() Oct 14 02:48:22.271153 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 14 02:48:22.271153 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rec.execute(self) Oct 14 02:48:22.271153 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 14 02:48:22.272243 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 14 02:48:22.272243 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 14 02:48:22.272243 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 14 02:48:22.272243 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 14 02:48:22.272243 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 14 02:48:22.272243 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 14 02:48:22.272243 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 14 02:48:22.272243 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 14 02:48:22.272243 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 14 02:48:22.272243 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 14 02:48:22.272243 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 14 02:48:22.272243 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 14 02:48:22.272243 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 14 02:48:22.272243 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 14 02:48:22.272243 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 14 02:48:22.273171 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 14 02:48:22.273171 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception Oct 14 02:48:22.273171 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 14 02:48:22.273171 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 14 02:48:22.273171 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 14 02:48:22.273171 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 14 02:48:22.273171 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 14 02:48:22.273171 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application result = self._query(query) Oct 14 02:48:22.273171 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 14 02:48:22.273171 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application conn.query(q) Oct 14 02:48:22.273171 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 14 02:48:22.273171 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 14 02:48:22.273171 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 14 02:48:22.273171 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application result.read() Oct 14 02:48:22.273171 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 14 02:48:22.273171 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 14 02:48:22.273171 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 14 02:48:22.274385 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 14 02:48:22.274385 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 14 02:48:22.274385 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 14 02:48:22.274385 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 14 02:48:22.274385 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 14 02:48:22.274385 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 14 02:48:22.274385 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 14 02:48:22.274385 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [parameters: {'id': '3b3e7d2a6a9b427eb4d1b19475bf8aa8', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 14 02:48:22.274385 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 14 02:48:22.274385 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:48:22.274385 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 02:48:22.274385 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:48:22.274385 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:48:22.274385 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 14 02:48:22.274385 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 14 02:48:22.275416 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 14 02:48:22.275416 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 14 02:48:22.275416 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 14 02:48:22.275416 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:48:22.275416 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 02:48:22.275416 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:48:22.275416 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:48:22.275416 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:48:22.275416 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:48:22.275416 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:48:22.275416 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:48:22.275416 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:48:22.275416 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:48:22.275416 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:48:22.275416 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:48:22.275416 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:48:22.275416 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:48:22.276456 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 14 02:48:22.276456 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 14 02:48:22.276456 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:48:22.276456 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:48:22.276456 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 14 02:48:22.276456 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 14 02:48:22.276456 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:48:22.276456 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:48:22.276456 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 14 02:48:22.276456 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 14 02:48:22.276456 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 14 02:48:22.276456 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:48:22.276456 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 8/16] 10.222.0.39 () {60 vars in 1137 bytes} [Fri Oct 14 02:48:22 2022] POST /identity/v3/domains => generated 190 bytes in 223 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 14 02:48:22.276456 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d3e7e7-6cb9-41bb-ab07-a61656c76968 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:22.323589 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fc4c55-9255-4b7a-b1a3-887ea2d0573d None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:22.355936 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743915b9-d438-4caa-ba61-00fb134a3275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:22.356562 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743915b9-d438-4caa-ba61-00fb134a3275 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:22.356842 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743915b9-d438-4caa-ba61-00fb134a3275 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:22.357122 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743915b9-d438-4caa-ba61-00fb134a3275 None admin] PATH_INFO: `/v3/users/316b39adbf9f4b0d9c13dc4c11a24e73` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:22.358285 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743915b9-d438-4caa-ba61-00fb134a3275 None admin] RBAC: Authorizing `identity:get_user(user_id=316b39adbf9f4b0d9c13dc4c11a24e73)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:22.359751 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743915b9-d438-4caa-ba61-00fb134a3275 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:22.361507 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 9/17] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Oct 14 02:48:22 2022] GET /identity/v3/users/316b39adbf9f4b0d9c13dc4c11a24e73 => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:48:22.366839 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743915b9-d438-4caa-ba61-00fb134a3275 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:22.371601 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d3e7e7-6cb9-41bb-ab07-a61656c76968 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:22.382775 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:22.405125 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8838dd6c-f6b9-4e1f-9d6e-ed6d534c5ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:22.406012 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8838dd6c-f6b9-4e1f-9d6e-ed6d534c5ad5 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:22.406295 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8838dd6c-f6b9-4e1f-9d6e-ed6d534c5ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:22.406568 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8838dd6c-f6b9-4e1f-9d6e-ed6d534c5ad5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:22.423937 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8838dd6c-f6b9-4e1f-9d6e-ed6d534c5ad5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:22.425455 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8838dd6c-f6b9-4e1f-9d6e-ed6d534c5ad5 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:22.441542 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-8838dd6c-f6b9-4e1f-9d6e-ed6d534c5ad5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:48:22.441542 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:48:22.441542 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:48:22.441542 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:48:22.441542 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:48:22.441542 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:48:22.441542 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:48:22.441542 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:48:22.441542 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:48:22.441542 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:48:22.441542 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:48:22.441542 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:48:22.441542 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:48:22.441542 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:48:22.441542 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:48:22.441542 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:48:22.442579 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:48:22.442579 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:48:22.442579 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:48:22.442579 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:48:22.442579 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:48:22.442579 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:48:22.442579 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:48:22.442579 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:48:22.442579 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:48:22.442579 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:48:22.442579 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:48:22.442579 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:48:22.442579 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:48:22.442579 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:48:22.442579 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:48:22.442579 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:48:22.442579 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:48:22.443471 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:48:22.443471 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:48:22.443471 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:48:22.443471 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 9/18] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:48:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 02:48:22.449202 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8838dd6c-f6b9-4e1f-9d6e-ed6d534c5ad5 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:22.466629 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743915b9-d438-4caa-ba61-00fb134a3275 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:22.470566 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:22.499392 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1763e3a2-970d-475f-b2dd-0e13a7b696b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:22.500012 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1763e3a2-970d-475f-b2dd-0e13a7b696b6 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:22.500291 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1763e3a2-970d-475f-b2dd-0e13a7b696b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:22.500563 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1763e3a2-970d-475f-b2dd-0e13a7b696b6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:22.501678 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1763e3a2-970d-475f-b2dd-0e13a7b696b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:22.503166 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1763e3a2-970d-475f-b2dd-0e13a7b696b6 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:22.504706 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 10/19] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:48:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:48:22.511028 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1763e3a2-970d-475f-b2dd-0e13a7b696b6 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:22.548985 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8838dd6c-f6b9-4e1f-9d6e-ed6d534c5ad5 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:22.582673 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12363cd-6c88-4b8c-b213-7c75ea747d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:22.583214 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12363cd-6c88-4b8c-b213-7c75ea747d57 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:22.583501 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12363cd-6c88-4b8c-b213-7c75ea747d57 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:22.583781 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12363cd-6c88-4b8c-b213-7c75ea747d57 None admin] PATH_INFO: `/v3/domains` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:22.584427 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12363cd-6c88-4b8c-b213-7c75ea747d57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:48:22.584733 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12363cd-6c88-4b8c-b213-7c75ea747d57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:22.586149 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12363cd-6c88-4b8c-b213-7c75ea747d57 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:22.590331 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:22.602738 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 10/20] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Oct 14 02:48:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:48:22.609027 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1763e3a2-970d-475f-b2dd-0e13a7b696b6 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:22.646990 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27237b2-4f5b-4640-abbf-0062c6ec281a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:22.647669 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27237b2-4f5b-4640-abbf-0062c6ec281a None admin] REQUEST_METHOD: `PUT` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:22.647956 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27237b2-4f5b-4640-abbf-0062c6ec281a None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:22.648242 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27237b2-4f5b-4640-abbf-0062c6ec281a None admin] PATH_INFO: `/v3/domains/default/users/316b39adbf9f4b0d9c13dc4c11a24e73/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:22.650287 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27237b2-4f5b-4640-abbf-0062c6ec281a None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=316b39adbf9f4b0d9c13dc4c11a24e73, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:22.651775 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27237b2-4f5b-4640-abbf-0062c6ec281a None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:22.839546 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 11/21] 10.222.0.39 () {58 vars in 1361 bytes} [Fri Oct 14 02:48:22 2022] PUT /identity/v3/domains/default/users/316b39adbf9f4b0d9c13dc4c11a24e73/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 02:48:25.092588 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fd9d20-4f67-4d2f-baf0-129b96f602e8 None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:25.093118 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fd9d20-4f67-4d2f-baf0-129b96f602e8 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:25.093118 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fd9d20-4f67-4d2f-baf0-129b96f602e8 None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:25.094659 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 11/22] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:48:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:48:25.100352 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fb6b2f-8c7c-48f0-892f-c12db160d366 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:25.101258 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fb6b2f-8c7c-48f0-892f-c12db160d366 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:25.101817 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fb6b2f-8c7c-48f0-892f-c12db160d366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:25.127173 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-14fb6b2f-8c7c-48f0-892f-c12db160d366 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:48:25.129173 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-14fb6b2f-8c7c-48f0-892f-c12db160d366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:48:25.211749 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 12/23] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:48:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:48:25.289952 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03750e2b-da53-43f9-9662-2a65cd8bf0da None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:25.290552 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03750e2b-da53-43f9-9662-2a65cd8bf0da None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:25.291026 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03750e2b-da53-43f9-9662-2a65cd8bf0da None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:25.292673 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 12/24] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:48:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:48:25.299038 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ccc218-283c-455a-a9e6-35f63181a732 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:25.299702 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ccc218-283c-455a-a9e6-35f63181a732 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:25.300183 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ccc218-283c-455a-a9e6-35f63181a732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:25.333580 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-95ccc218-283c-455a-a9e6-35f63181a732 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:48:25.335308 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-95ccc218-283c-455a-a9e6-35f63181a732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:48:25.422995 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:25.427881 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 13/25] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:48:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 14 02:48:25.496984 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e7f27-028f-4ec9-bf51-ca2ab5ed424f None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:25.497299 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e7f27-028f-4ec9-bf51-ca2ab5ed424f None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:25.497549 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e7f27-028f-4ec9-bf51-ca2ab5ed424f None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:25.498919 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 13/26] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:48:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:48:25.504230 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcbcdb9-62c2-4903-b59f-f041fc85d86f None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:25.504799 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcbcdb9-62c2-4903-b59f-f041fc85d86f None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:25.505360 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcbcdb9-62c2-4903-b59f-f041fc85d86f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:25.535370 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-5fcbcdb9-62c2-4903-b59f-f041fc85d86f None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:48:25.537002 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-5fcbcdb9-62c2-4903-b59f-f041fc85d86f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:48:25.633744 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 14/27] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:48:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:48:25.641961 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981e7f27-028f-4ec9-bf51-ca2ab5ed424f None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:25.755851 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9a9860-6631-4f4c-b69e-29f657a84726 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:25.756187 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9a9860-6631-4f4c-b69e-29f657a84726 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:25.756431 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9a9860-6631-4f4c-b69e-29f657a84726 None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:25.759175 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:25.767232 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 15/28] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:48:25 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:48:25.780578 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a8bb63-a962-4b68-98bb-4819cf928373 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:25.780868 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a8bb63-a962-4b68-98bb-4819cf928373 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:25.781155 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a8bb63-a962-4b68-98bb-4819cf928373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:25.794186 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981e7f27-028f-4ec9-bf51-ca2ab5ed424f None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:25.819223 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-d1a8bb63-a962-4b68-98bb-4819cf928373 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:48:25.820503 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-d1a8bb63-a962-4b68-98bb-4819cf928373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:48:25.868264 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992c7c1d-1ba7-41f7-8dc9-8cb7f731c7d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:25.868908 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992c7c1d-1ba7-41f7-8dc9-8cb7f731c7d1 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:25.869192 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992c7c1d-1ba7-41f7-8dc9-8cb7f731c7d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:25.869467 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992c7c1d-1ba7-41f7-8dc9-8cb7f731c7d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:25.887409 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992c7c1d-1ba7-41f7-8dc9-8cb7f731c7d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:25.888842 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992c7c1d-1ba7-41f7-8dc9-8cb7f731c7d1 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:25.913840 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-992c7c1d-1ba7-41f7-8dc9-8cb7f731c7d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:48:25.913840 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:48:25.913840 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:48:25.913840 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:48:25.913840 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:48:25.913840 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:48:25.913840 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:48:25.913840 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:48:25.913840 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:48:25.913840 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:48:25.913840 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:48:25.913840 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:48:25.913840 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:48:25.913840 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:48:25.913840 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:48:25.913840 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:48:25.915232 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:48:25.915232 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:48:25.915232 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:48:25.915232 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:48:25.915232 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:48:25.915232 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:48:25.915232 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:48:25.915232 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:48:25.915232 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:48:25.915232 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:48:25.915232 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:48:25.915232 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:48:25.915232 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:48:25.915232 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:48:25.915232 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:48:25.915232 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:48:25.915232 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:48:25.916043 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:48:25.916043 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:48:25.916043 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:48:25.916908 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 14/29] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:48:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:48:25.918252 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992c7c1d-1ba7-41f7-8dc9-8cb7f731c7d1 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:25.935758 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 16/30] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:48:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 14 02:48:25.936415 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a8bb63-a962-4b68-98bb-4819cf928373 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:26.062141 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:26.127202 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992c7c1d-1ba7-41f7-8dc9-8cb7f731c7d1 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:26.138745 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a8bb63-a962-4b68-98bb-4819cf928373 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:26.165166 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:26.170663 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067a84a0-dbcc-486c-a13c-3e2cb645d415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:26.171491 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067a84a0-dbcc-486c-a13c-3e2cb645d415 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:26.171918 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067a84a0-dbcc-486c-a13c-3e2cb645d415 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:26.172384 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067a84a0-dbcc-486c-a13c-3e2cb645d415 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:26.184536 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067a84a0-dbcc-486c-a13c-3e2cb645d415 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:48:26.191101 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-067a84a0-dbcc-486c-a13c-3e2cb645d415 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:48:26.191101 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:48:26.191101 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:48:26.191101 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:48:26.191101 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:48:26.191101 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:48:26.191101 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:48:26.191101 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:48:26.191101 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:48:26.191101 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:48:26.191101 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:48:26.191101 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:48:26.191101 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:48:26.191101 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:48:26.191101 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:48:26.191101 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:48:26.192108 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:48:26.192108 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:48:26.192108 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:48:26.192108 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:48:26.192108 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:48:26.192108 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:48:26.192108 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:48:26.192108 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:48:26.192108 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:48:26.192108 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:48:26.192108 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:48:26.192108 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:48:26.192108 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:48:26.192108 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:48:26.192108 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:48:26.192108 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:48:26.193066 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:48:26.193066 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:48:26.193066 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:48:26.193066 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:48:26.193066 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:48:26.193066 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:48:26.193066 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:48:26.193066 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:48:26.193066 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:48:26.193066 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:48:26.193066 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:48:26.193066 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:48:26.195354 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 15/31] 10.222.0.39 () {56 vars in 1096 bytes} [Fri Oct 14 02:48:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:48:26.196314 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067a84a0-dbcc-486c-a13c-3e2cb645d415 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:26.250479 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52433e81-b749-481c-838b-6b9609e9a842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:26.251111 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52433e81-b749-481c-838b-6b9609e9a842 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:26.251405 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52433e81-b749-481c-838b-6b9609e9a842 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:26.251689 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52433e81-b749-481c-838b-6b9609e9a842 None admin] PATH_INFO: `/v3/domains` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:26.252352 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52433e81-b749-481c-838b-6b9609e9a842 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:48:26.252862 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52433e81-b749-481c-838b-6b9609e9a842 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:26.254345 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52433e81-b749-481c-838b-6b9609e9a842 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:26.280237 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 17/32] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Oct 14 02:48:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:48:26.281245 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52433e81-b749-481c-838b-6b9609e9a842 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:26.310398 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067a84a0-dbcc-486c-a13c-3e2cb645d415 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:26.345840 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29d7890-49e2-4a45-a9a4-78ed906b2f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:26.346685 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d7890-49e2-4a45-a9a4-78ed906b2f1f None admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:26.347110 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d7890-49e2-4a45-a9a4-78ed906b2f1f None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:26.347558 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d7890-49e2-4a45-a9a4-78ed906b2f1f None admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:26.348236 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29d7890-49e2-4a45-a9a4-78ed906b2f1f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:26.349640 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29d7890-49e2-4a45-a9a4-78ed906b2f1f None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:26.395475 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52433e81-b749-481c-838b-6b9609e9a842 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:26.416342 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:26.430453 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1886326e-4a23-4a0c-ba7f-92f59b8d39dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:26.431062 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1886326e-4a23-4a0c-ba7f-92f59b8d39dd None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:26.431344 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1886326e-4a23-4a0c-ba7f-92f59b8d39dd None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:26.431617 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1886326e-4a23-4a0c-ba7f-92f59b8d39dd None admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:26.432273 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1886326e-4a23-4a0c-ba7f-92f59b8d39dd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:48:26.432602 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1886326e-4a23-4a0c-ba7f-92f59b8d39dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:26.434022 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1886326e-4a23-4a0c-ba7f-92f59b8d39dd None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:26.454396 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 18/33] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Oct 14 02:48:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:48:26.455287 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1886326e-4a23-4a0c-ba7f-92f59b8d39dd None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:26.567958 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1886326e-4a23-4a0c-ba7f-92f59b8d39dd None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:26.587204 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 16/34] 10.222.0.39 () {60 vars in 1131 bytes} [Fri Oct 14 02:48:26 2022] POST /identity/v3/roles => generated 219 bytes in 386 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:48:26.587204 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29d7890-49e2-4a45-a9a4-78ed906b2f1f None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:26.641788 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9462be5-6cf6-4d28-9031-4cdc71418fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:26.642413 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9462be5-6cf6-4d28-9031-4cdc71418fd9 None admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:26.642955 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9462be5-6cf6-4d28-9031-4cdc71418fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:26.643230 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9462be5-6cf6-4d28-9031-4cdc71418fd9 None admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:26.643907 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9462be5-6cf6-4d28-9031-4cdc71418fd9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:26.645313 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9462be5-6cf6-4d28-9031-4cdc71418fd9 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:26.680867 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:26.719294 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 19/35] 10.222.0.39 () {60 vars in 1131 bytes} [Fri Oct 14 02:48:26 2022] POST /identity/v3/roles => generated 225 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:48:26.720503 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9462be5-6cf6-4d28-9031-4cdc71418fd9 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:26.750877 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29d7890-49e2-4a45-a9a4-78ed906b2f1f None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:26.796794 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abde53f-51d5-4993-bde1-86360a716501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:26.797858 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abde53f-51d5-4993-bde1-86360a716501 None admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:26.798354 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abde53f-51d5-4993-bde1-86360a716501 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:26.798833 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abde53f-51d5-4993-bde1-86360a716501 None admin] PATH_INFO: `/v3/projects` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:26.799794 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abde53f-51d5-4993-bde1-86360a716501 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:26.801841 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abde53f-51d5-4993-bde1-86360a716501 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:26.919509 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:26.923089 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 17/36] 10.222.0.39 () {60 vars in 1141 bytes} [Fri Oct 14 02:48:26 2022] POST /identity/v3/projects => generated 301 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:48:26.924760 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82abeaf7-f479-4c79-b094-f9dff54df55c None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:26.925066 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82abeaf7-f479-4c79-b094-f9dff54df55c None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:26.925334 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82abeaf7-f479-4c79-b094-f9dff54df55c None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:26.927060 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 18/37] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:48:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:48:26.941145 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0552433a-9060-424a-9b27-f3bf33a0bed0 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:26.941342 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0552433a-9060-424a-9b27-f3bf33a0bed0 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:26.941621 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0552433a-9060-424a-9b27-f3bf33a0bed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:26.969594 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-0552433a-9060-424a-9b27-f3bf33a0bed0 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:48:26.971371 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-0552433a-9060-424a-9b27-f3bf33a0bed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:48:27.027468 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9462be5-6cf6-4d28-9031-4cdc71418fd9 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:27.068328 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f762136-1857-4a93-8c51-3d788d60842b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:27.068816 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 19/38] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:48:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 14 02:48:27.068980 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f762136-1857-4a93-8c51-3d788d60842b None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:27.069271 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f762136-1857-4a93-8c51-3d788d60842b None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:27.069614 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f762136-1857-4a93-8c51-3d788d60842b None admin] PATH_INFO: `/v3/users/316b39adbf9f4b0d9c13dc4c11a24e73` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:27.071539 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f762136-1857-4a93-8c51-3d788d60842b None admin] RBAC: Authorizing `identity:get_user(user_id=316b39adbf9f4b0d9c13dc4c11a24e73)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:27.073053 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f762136-1857-4a93-8c51-3d788d60842b None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:27.076101 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 20/39] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Oct 14 02:48:26 2022] GET /identity/v3/users/316b39adbf9f4b0d9c13dc4c11a24e73 => generated 247 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:48:27.080502 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0552433a-9060-424a-9b27-f3bf33a0bed0 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:27.118364 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:27.196447 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0552433a-9060-424a-9b27-f3bf33a0bed0 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:27.236505 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3084f6-46c9-43a8-a1de-d79fd6d4205a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:27.237148 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3084f6-46c9-43a8-a1de-d79fd6d4205a None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:27.237489 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3084f6-46c9-43a8-a1de-d79fd6d4205a None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:27.237796 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3084f6-46c9-43a8-a1de-d79fd6d4205a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:27.239009 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3084f6-46c9-43a8-a1de-d79fd6d4205a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:27.240521 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3084f6-46c9-43a8-a1de-d79fd6d4205a None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:27.242143 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 20/40] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:48:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:48:27.248782 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f762136-1857-4a93-8c51-3d788d60842b None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:27.307732 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09c03bf-705c-41b7-9202-187e8e98958a None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:27.308013 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09c03bf-705c-41b7-9202-187e8e98958a None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:27.308298 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09c03bf-705c-41b7-9202-187e8e98958a None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:27.309815 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 21/41] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:48:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 14 02:48:27.314507 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dc2676-3519-4cb4-b6bc-ef09d91e3a94 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:27.314802 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dc2676-3519-4cb4-b6bc-ef09d91e3a94 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:27.315083 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dc2676-3519-4cb4-b6bc-ef09d91e3a94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:27.332858 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-c3dc2676-3519-4cb4-b6bc-ef09d91e3a94 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:48:27.334093 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-c3dc2676-3519-4cb4-b6bc-ef09d91e3a94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:48:27.366015 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f762136-1857-4a93-8c51-3d788d60842b None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:27.376974 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:27.399677 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8067d37a-efa0-4ca0-b036-f8912b6216b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:27.400290 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8067d37a-efa0-4ca0-b036-f8912b6216b4 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:27.400577 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8067d37a-efa0-4ca0-b036-f8912b6216b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:27.400889 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8067d37a-efa0-4ca0-b036-f8912b6216b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:27.401663 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8067d37a-efa0-4ca0-b036-f8912b6216b4 None admin] RBAC: Adding query filter params (role.id=9e86d182c0594e0785b72ba5f390a114, scope.domain.id=default, user.id=316b39adbf9f4b0d9c13dc4c11a24e73) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:48:27.402006 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8067d37a-efa0-4ca0-b036-f8912b6216b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:27.403459 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8067d37a-efa0-4ca0-b036-f8912b6216b4 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:27.404914 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 22/42] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:48:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 14 02:48:27.405743 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dc2676-3519-4cb4-b6bc-ef09d91e3a94 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:27.410804 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 21/43] 10.222.0.39 () {56 vars in 1322 bytes} [Fri Oct 14 02:48:27 2022] GET /identity/v3/role_assignments?user.id=316b39adbf9f4b0d9c13dc4c11a24e73&scope.domain.id=default&role.id=9e86d182c0594e0785b72ba5f390a114 => generated 530 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:48:27.443011 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:27.519621 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dc2676-3519-4cb4-b6bc-ef09d91e3a94 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:27.556856 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2376a72-ff71-42ae-b73d-7ac2f0e6792f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:27.557489 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2376a72-ff71-42ae-b73d-7ac2f0e6792f None admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:27.557797 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2376a72-ff71-42ae-b73d-7ac2f0e6792f None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:27.558082 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2376a72-ff71-42ae-b73d-7ac2f0e6792f None admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:27.558842 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2376a72-ff71-42ae-b73d-7ac2f0e6792f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:27.560269 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2376a72-ff71-42ae-b73d-7ac2f0e6792f None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:27.660793 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 23/44] 10.222.0.39 () {60 vars in 1131 bytes} [Fri Oct 14 02:48:27 2022] POST /identity/v3/roles => generated 223 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:48:27.674614 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8067d37a-efa0-4ca0-b036-f8912b6216b4 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:27.779676 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8067d37a-efa0-4ca0-b036-f8912b6216b4 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:27.814920 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d9b9bd-9fef-4d6d-af64-33331db241b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:27.815556 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d9b9bd-9fef-4d6d-af64-33331db241b4 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:27.815837 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d9b9bd-9fef-4d6d-af64-33331db241b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:27.816112 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d9b9bd-9fef-4d6d-af64-33331db241b4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:27.817226 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d9b9bd-9fef-4d6d-af64-33331db241b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:27.818728 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d9b9bd-9fef-4d6d-af64-33331db241b4 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:27.822367 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 22/45] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:48:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:48:27.826141 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2376a72-ff71-42ae-b73d-7ac2f0e6792f None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:27.918672 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:27.927018 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2376a72-ff71-42ae-b73d-7ac2f0e6792f None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:27.959391 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6456ce-bb52-4e2d-8e5f-27296d7bf0ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:27.959999 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6456ce-bb52-4e2d-8e5f-27296d7bf0ac None admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:27.960285 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6456ce-bb52-4e2d-8e5f-27296d7bf0ac None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:27.960563 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6456ce-bb52-4e2d-8e5f-27296d7bf0ac None admin] PATH_INFO: `/v3/projects` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:27.961897 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6456ce-bb52-4e2d-8e5f-27296d7bf0ac None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:27.963343 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6456ce-bb52-4e2d-8e5f-27296d7bf0ac None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:28.043506 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 24/46] 10.222.0.39 () {60 vars in 1141 bytes} [Fri Oct 14 02:48:27 2022] POST /identity/v3/projects => generated 312 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:48:30.183750 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1dd0e4-0dbb-4862-ba96-43d36dc28c75 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:30.184258 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1dd0e4-0dbb-4862-ba96-43d36dc28c75 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:30.184342 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1dd0e4-0dbb-4862-ba96-43d36dc28c75 None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:30.185657 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 23/47] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:48:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:48:30.190847 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3538116-020f-4c89-b00f-a0a9a64a8f4b None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:30.191210 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3538116-020f-4c89-b00f-a0a9a64a8f4b None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:30.191412 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3538116-020f-4c89-b00f-a0a9a64a8f4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:30.209333 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-b3538116-020f-4c89-b00f-a0a9a64a8f4b None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:48:30.210648 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-b3538116-020f-4c89-b00f-a0a9a64a8f4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:48:30.274803 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 25/48] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:48:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:48:30.546043 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1dd0e4-0dbb-4862-ba96-43d36dc28c75 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:30.588939 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:30.644640 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1dd0e4-0dbb-4862-ba96-43d36dc28c75 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:30.677672 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cec5cd-d15b-49c9-a3ad-bf9227f470a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:30.678314 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cec5cd-d15b-49c9-a3ad-bf9227f470a0 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:30.678599 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cec5cd-d15b-49c9-a3ad-bf9227f470a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:30.678877 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cec5cd-d15b-49c9-a3ad-bf9227f470a0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:30.679999 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cec5cd-d15b-49c9-a3ad-bf9227f470a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:30.681460 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cec5cd-d15b-49c9-a3ad-bf9227f470a0 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:30.683043 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 24/49] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:48:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:48:30.688575 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3538116-020f-4c89-b00f-a0a9a64a8f4b None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:30.759716 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:30.784252 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3538116-020f-4c89-b00f-a0a9a64a8f4b None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:30.817332 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcdb067-efbf-4350-acb3-4e50b278971e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:30.817909 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcdb067-efbf-4350-acb3-4e50b278971e None admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:30.818198 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcdb067-efbf-4350-acb3-4e50b278971e None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:30.818477 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcdb067-efbf-4350-acb3-4e50b278971e None admin] PATH_INFO: `/v3/projects` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:30.819125 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcdb067-efbf-4350-acb3-4e50b278971e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:30.820620 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcdb067-efbf-4350-acb3-4e50b278971e None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:30.883655 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 26/50] 10.222.0.39 () {60 vars in 1140 bytes} [Fri Oct 14 02:48:30 2022] POST /identity/v3/projects => generated 298 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:48:33.016949 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc662b8e-c0f0-4ed4-840a-4c7e48072ab5 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:33.017333 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc662b8e-c0f0-4ed4-840a-4c7e48072ab5 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:33.017496 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc662b8e-c0f0-4ed4-840a-4c7e48072ab5 None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:33.018914 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 25/51] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:48:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:48:33.024619 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5c56ff-552a-40b1-8c31-87d4a92fdea9 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:33.025008 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5c56ff-552a-40b1-8c31-87d4a92fdea9 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:33.025183 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5c56ff-552a-40b1-8c31-87d4a92fdea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:33.043578 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-4c5c56ff-552a-40b1-8c31-87d4a92fdea9 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:48:33.044777 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-4c5c56ff-552a-40b1-8c31-87d4a92fdea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:48:33.108899 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 27/52] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:48:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:48:33.378052 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc662b8e-c0f0-4ed4-840a-4c7e48072ab5 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:33.476170 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc662b8e-c0f0-4ed4-840a-4c7e48072ab5 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:33.486640 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:33.509454 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7966711f-ff0d-45a8-b152-7c0e2cca3edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:33.510105 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7966711f-ff0d-45a8-b152-7c0e2cca3edf None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:33.510386 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7966711f-ff0d-45a8-b152-7c0e2cca3edf None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:33.510665 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7966711f-ff0d-45a8-b152-7c0e2cca3edf None admin] PATH_INFO: `/v3/domains/default` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:33.511775 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7966711f-ff0d-45a8-b152-7c0e2cca3edf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:33.513239 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7966711f-ff0d-45a8-b152-7c0e2cca3edf None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:33.514813 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 26/53] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:48:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:48:33.520226 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5c56ff-552a-40b1-8c31-87d4a92fdea9 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:33.573884 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:33.614358 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5c56ff-552a-40b1-8c31-87d4a92fdea9 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:33.648020 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267761b8-bf4a-47b3-99c0-efff6031288e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:33.648594 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267761b8-bf4a-47b3-99c0-efff6031288e None admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:33.648878 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267761b8-bf4a-47b3-99c0-efff6031288e None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:33.649152 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267761b8-bf4a-47b3-99c0-efff6031288e None admin] PATH_INFO: `/v3/users` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:33.649814 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267761b8-bf4a-47b3-99c0-efff6031288e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:33.651575 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267761b8-bf4a-47b3-99c0-efff6031288e None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:33.848146 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 28/54] 10.222.0.39 () {60 vars in 1132 bytes} [Fri Oct 14 02:48:33 2022] POST /identity/v3/users => generated 275 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:48:37.146342 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e1ef59-2ea2-454a-aa4e-828645de7433 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:37.147352 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e1ef59-2ea2-454a-aa4e-828645de7433 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:37.147841 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e1ef59-2ea2-454a-aa4e-828645de7433 None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:37.149469 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 27/55] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:48:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:48:37.154785 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff03f666-8738-4661-9557-865d8bade728 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:37.155623 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff03f666-8738-4661-9557-865d8bade728 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:37.156101 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff03f666-8738-4661-9557-865d8bade728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:37.181316 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-ff03f666-8738-4661-9557-865d8bade728 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:48:37.182736 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-ff03f666-8738-4661-9557-865d8bade728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:48:37.259767 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c512092-a0bd-44d3-9318-d4edd7318a1e None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:37.260497 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c512092-a0bd-44d3-9318-d4edd7318a1e None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:37.260988 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c512092-a0bd-44d3-9318-d4edd7318a1e None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:37.262718 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 28/56] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:48:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:48:37.265875 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 29/57] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:48:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:48:37.268705 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1daf28-65b8-4f4b-bad4-8f816d817c46 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:37.269000 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1daf28-65b8-4f4b-bad4-8f816d817c46 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:37.269277 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1daf28-65b8-4f4b-bad4-8f816d817c46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:37.297016 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-da1daf28-65b8-4f4b-bad4-8f816d817c46 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:48:37.302498 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-da1daf28-65b8-4f4b-bad4-8f816d817c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:48:37.406218 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caeef21-4423-47b2-a908-605d207e5257 None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:37.406875 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caeef21-4423-47b2-a908-605d207e5257 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:37.407155 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 29/58] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:48:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 14 02:48:37.407602 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caeef21-4423-47b2-a908-605d207e5257 None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:37.409246 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 30/59] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:48:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 14 02:48:37.415274 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86393d76-1d61-40de-8fe7-a0f9aeceecfe None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:37.415565 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86393d76-1d61-40de-8fe7-a0f9aeceecfe None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:37.415847 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86393d76-1d61-40de-8fe7-a0f9aeceecfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:37.434315 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-86393d76-1d61-40de-8fe7-a0f9aeceecfe None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:48:37.435543 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-86393d76-1d61-40de-8fe7-a0f9aeceecfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:48:37.511999 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 30/60] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:48:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:48:37.556315 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17c8194-6881-42fc-84e7-757728283a11 None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:37.556960 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17c8194-6881-42fc-84e7-757728283a11 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:37.557437 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17c8194-6881-42fc-84e7-757728283a11 None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:37.559019 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 31/61] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:48:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:48:37.564334 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367869c2-4f81-4e20-8123-f3c97a5fbd34 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:37.564849 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367869c2-4f81-4e20-8123-f3c97a5fbd34 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:37.565339 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367869c2-4f81-4e20-8123-f3c97a5fbd34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:37.583642 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-367869c2-4f81-4e20-8123-f3c97a5fbd34 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:48:37.585098 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-367869c2-4f81-4e20-8123-f3c97a5fbd34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:48:37.621751 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:37.667144 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 31/62] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:48:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:48:37.702436 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17c8194-6881-42fc-84e7-757728283a11 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:37.762889 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:37.832447 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17c8194-6881-42fc-84e7-757728283a11 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:37.855332 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367869c2-4f81-4e20-8123-f3c97a5fbd34 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:37.875232 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f866f79-7816-409a-b2ba-2c5e8bed6afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:37.875909 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f866f79-7816-409a-b2ba-2c5e8bed6afc None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:37.876192 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f866f79-7816-409a-b2ba-2c5e8bed6afc None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:37.876469 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f866f79-7816-409a-b2ba-2c5e8bed6afc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:37.887838 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f866f79-7816-409a-b2ba-2c5e8bed6afc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:48:37.893061 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-3f866f79-7816-409a-b2ba-2c5e8bed6afc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 14 02:48:37.893061 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:48:37.893061 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:48:37.893061 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:48:37.893061 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:48:37.893061 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:48:37.893061 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:48:37.893061 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:48:37.893061 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:48:37.893061 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:48:37.893061 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:48:37.893061 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:48:37.893061 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:48:37.893061 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:48:37.893061 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:48:37.893061 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:48:37.893921 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:48:37.893921 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:48:37.893921 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:48:37.893921 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:48:37.893921 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:48:37.893921 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:48:37.893921 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:48:37.893921 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:48:37.893921 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:48:37.893921 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:48:37.893921 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:48:37.893921 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:48:37.893921 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:48:37.893921 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:48:37.893921 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:48:37.893921 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:48:37.894688 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:48:37.894688 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:48:37.894688 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:48:37.894688 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:48:37.894688 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:48:37.894688 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:48:37.894688 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:48:37.894688 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:48:37.894688 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:48:37.894688 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:48:37.894688 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 14 02:48:37.894688 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:48:37.894688 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 32/63] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Oct 14 02:48:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:48:37.895294 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f866f79-7816-409a-b2ba-2c5e8bed6afc None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:37.962657 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367869c2-4f81-4e20-8123-f3c97a5fbd34 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:37.997616 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aabc50-c8d1-4033-ada0-85469466ee79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:37.998498 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aabc50-c8d1-4033-ada0-85469466ee79 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:37.998804 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aabc50-c8d1-4033-ada0-85469466ee79 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:37.999062 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aabc50-c8d1-4033-ada0-85469466ee79 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:38.011110 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16aabc50-c8d1-4033-ada0-85469466ee79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:48:38.011456 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f866f79-7816-409a-b2ba-2c5e8bed6afc None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:38.013202 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:38.015987 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-16aabc50-c8d1-4033-ada0-85469466ee79 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 14 02:48:38.015987 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:48:38.015987 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:48:38.015987 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:48:38.015987 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:48:38.015987 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:48:38.015987 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:48:38.015987 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:48:38.015987 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:48:38.015987 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:48:38.015987 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:48:38.015987 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:48:38.015987 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:48:38.015987 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:48:38.015987 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:48:38.015987 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:48:38.016827 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:48:38.016827 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:48:38.016827 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:48:38.016827 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:48:38.016827 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:48:38.016827 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:48:38.016827 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:48:38.016827 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:48:38.016827 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:48:38.016827 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:48:38.016827 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:48:38.016827 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:48:38.016827 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:48:38.016827 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:48:38.016827 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:48:38.016827 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:48:38.025670 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:48:38.025670 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:48:38.025670 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:48:38.025670 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:48:38.025670 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:48:38.025670 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:48:38.025670 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:48:38.025670 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:48:38.025670 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:48:38.025670 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:48:38.025670 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 14 02:48:38.025670 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:48:38.025670 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 32/64] 10.222.0.39 () {56 vars in 1114 bytes} [Fri Oct 14 02:48:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 02:48:38.025670 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b5238c-a24b-4fc7-9bc8-29400266f604 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:38.031679 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b5238c-a24b-4fc7-9bc8-29400266f604 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:38.031679 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b5238c-a24b-4fc7-9bc8-29400266f604 None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:38.031679 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 33/65] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:48:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:48:38.031679 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b5238c-a24b-4fc7-9bc8-29400266f604 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:38.052875 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:38.054931 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7414ea-0a0b-46e0-af07-839692cb7646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:38.055573 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7414ea-0a0b-46e0-af07-839692cb7646 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:38.055855 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7414ea-0a0b-46e0-af07-839692cb7646 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:38.056129 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7414ea-0a0b-46e0-af07-839692cb7646 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:38.066188 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7414ea-0a0b-46e0-af07-839692cb7646 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:48:38.071515 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-3d7414ea-0a0b-46e0-af07-839692cb7646 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:48:38.071515 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:48:38.071515 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:48:38.071515 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:48:38.071515 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:48:38.071515 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:48:38.071515 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:48:38.071515 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:48:38.071515 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:48:38.071515 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:48:38.071515 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:48:38.071515 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:48:38.071515 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:48:38.071515 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:48:38.071515 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:48:38.071515 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:48:38.072478 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:48:38.072478 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:48:38.072478 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:48:38.072478 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:48:38.072478 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:48:38.072478 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:48:38.072478 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:48:38.072478 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:48:38.072478 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:48:38.072478 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:48:38.072478 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:48:38.072478 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:48:38.072478 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:48:38.072478 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:48:38.072478 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:48:38.072478 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:48:38.073259 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:48:38.073259 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:48:38.073259 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:48:38.073259 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:48:38.073259 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:48:38.073259 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:48:38.073259 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:48:38.073259 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:48:38.073259 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:48:38.073259 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:48:38.073259 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:48:38.073259 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:48:38.073259 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 33/66] 10.222.0.39 () {56 vars in 1096 bytes} [Fri Oct 14 02:48:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:48:38.074100 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7414ea-0a0b-46e0-af07-839692cb7646 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:38.131743 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b5238c-a24b-4fc7-9bc8-29400266f604 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:38.166050 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd49f234-c9a8-4ed5-86db-a6187d5a7698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:38.166659 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd49f234-c9a8-4ed5-86db-a6187d5a7698 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:38.166929 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd49f234-c9a8-4ed5-86db-a6187d5a7698 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:38.167197 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd49f234-c9a8-4ed5-86db-a6187d5a7698 None admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:38.167872 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd49f234-c9a8-4ed5-86db-a6187d5a7698 None admin] RBAC: Adding query filter params (name=member) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:48:38.168177 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd49f234-c9a8-4ed5-86db-a6187d5a7698 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:38.169649 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd49f234-c9a8-4ed5-86db-a6187d5a7698 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:38.173318 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7414ea-0a0b-46e0-af07-839692cb7646 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:38.177614 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 34/67] 10.222.0.39 () {56 vars in 1101 bytes} [Fri Oct 14 02:48:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:48:38.178446 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd49f234-c9a8-4ed5-86db-a6187d5a7698 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:38.207496 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7893f47-b5c8-410b-9571-cd03456c6713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:38.208103 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7893f47-b5c8-410b-9571-cd03456c6713 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:38.208384 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7893f47-b5c8-410b-9571-cd03456c6713 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:38.208662 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7893f47-b5c8-410b-9571-cd03456c6713 None admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:38.209325 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7893f47-b5c8-410b-9571-cd03456c6713 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:48:38.209639 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7893f47-b5c8-410b-9571-cd03456c6713 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:38.211190 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7893f47-b5c8-410b-9571-cd03456c6713 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:38.223441 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 34/68] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Oct 14 02:48:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:48:38.224647 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3fcf49-8c8e-4284-9d25-09eeed33add7 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:38.225103 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3fcf49-8c8e-4284-9d25-09eeed33add7 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:38.225205 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3fcf49-8c8e-4284-9d25-09eeed33add7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:38.242764 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-0c3fcf49-8c8e-4284-9d25-09eeed33add7 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:48:38.243953 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-0c3fcf49-8c8e-4284-9d25-09eeed33add7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:48:38.261055 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:38.271068 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:38.278867 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd49f234-c9a8-4ed5-86db-a6187d5a7698 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:38.309731 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 35/69] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:48:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:48:38.310619 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3fcf49-8c8e-4284-9d25-09eeed33add7 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:38.313050 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2928b27f-8b47-4cf1-bef2-88e590f41f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:38.314580 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2928b27f-8b47-4cf1-bef2-88e590f41f56 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:38.314869 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2928b27f-8b47-4cf1-bef2-88e590f41f56 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:38.315150 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2928b27f-8b47-4cf1-bef2-88e590f41f56 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:38.329170 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2928b27f-8b47-4cf1-bef2-88e590f41f56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:48:38.336285 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-2928b27f-8b47-4cf1-bef2-88e590f41f56 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 14 02:48:38.336285 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:48:38.336285 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:48:38.336285 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:48:38.336285 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:48:38.336285 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:48:38.336285 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:48:38.336285 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:48:38.336285 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:48:38.336285 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:48:38.336285 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:48:38.336285 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:48:38.336285 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:48:38.336285 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:48:38.336285 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:48:38.336285 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:48:38.337098 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:48:38.337098 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:48:38.337098 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:48:38.337098 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:48:38.337098 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:48:38.337098 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:48:38.337098 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:48:38.337098 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:48:38.337098 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:48:38.337098 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:48:38.337098 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:48:38.337098 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:48:38.337098 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:48:38.337098 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:48:38.337098 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:48:38.337098 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:48:38.338251 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:48:38.338251 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:48:38.338251 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:48:38.338251 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:48:38.338251 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:48:38.338251 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:48:38.338251 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:48:38.338251 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:48:38.338251 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:48:38.338251 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:48:38.338251 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 14 02:48:38.338251 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:48:38.338826 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 35/70] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Oct 14 02:48:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:48:38.349156 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2928b27f-8b47-4cf1-bef2-88e590f41f56 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:38.414097 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3fcf49-8c8e-4284-9d25-09eeed33add7 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:38.449183 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc01439-0627-4faa-86c6-c1df8b61cff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:38.449823 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc01439-0627-4faa-86c6-c1df8b61cff3 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:38.450111 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc01439-0627-4faa-86c6-c1df8b61cff3 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:38.450383 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc01439-0627-4faa-86c6-c1df8b61cff3 None admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:38.451042 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc01439-0627-4faa-86c6-c1df8b61cff3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:48:38.451356 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc01439-0627-4faa-86c6-c1df8b61cff3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:38.452729 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc01439-0627-4faa-86c6-c1df8b61cff3 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:38.453240 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2928b27f-8b47-4cf1-bef2-88e590f41f56 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:38.461086 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 36/71] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Oct 14 02:48:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:48:38.463193 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc01439-0627-4faa-86c6-c1df8b61cff3 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:38.484682 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:38.489092 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35faf98-3bd4-449d-a47b-ff04cb599001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:38.489741 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35faf98-3bd4-449d-a47b-ff04cb599001 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:38.490053 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35faf98-3bd4-449d-a47b-ff04cb599001 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:38.490331 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35faf98-3bd4-449d-a47b-ff04cb599001 None admin] PATH_INFO: `/v3/users/2d06bb982e494934920981820968bc00` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:38.504432 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35faf98-3bd4-449d-a47b-ff04cb599001 None admin] RBAC: Authorizing `identity:get_user(user_id=2d06bb982e494934920981820968bc00)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:38.505947 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35faf98-3bd4-449d-a47b-ff04cb599001 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:38.508386 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 36/72] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Oct 14 02:48:38 2022] GET /identity/v3/users/2d06bb982e494934920981820968bc00 => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:48:38.508386 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35faf98-3bd4-449d-a47b-ff04cb599001 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:38.522068 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:38.573368 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc01439-0627-4faa-86c6-c1df8b61cff3 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:38.607283 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cf0954-52a3-479a-9a5f-ea65d35a44d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:38.607914 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cf0954-52a3-479a-9a5f-ea65d35a44d1 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:38.608197 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cf0954-52a3-479a-9a5f-ea65d35a44d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:38.608477 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cf0954-52a3-479a-9a5f-ea65d35a44d1 None admin] PATH_INFO: `/v3/users/2d06bb982e494934920981820968bc00` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:38.609879 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cf0954-52a3-479a-9a5f-ea65d35a44d1 None admin] RBAC: Authorizing `identity:get_user(user_id=2d06bb982e494934920981820968bc00)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:38.610305 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35faf98-3bd4-449d-a47b-ff04cb599001 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:38.611332 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cf0954-52a3-479a-9a5f-ea65d35a44d1 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:38.612920 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 37/73] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Oct 14 02:48:38 2022] GET /identity/v3/users/2d06bb982e494934920981820968bc00 => generated 275 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:48:38.613678 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cf0954-52a3-479a-9a5f-ea65d35a44d1 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:38.647251 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746144a5-b72f-4821-bf09-d25242c818fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:38.647854 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746144a5-b72f-4821-bf09-d25242c818fb None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:38.648159 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746144a5-b72f-4821-bf09-d25242c818fb None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:38.648432 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746144a5-b72f-4821-bf09-d25242c818fb None admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:38.649084 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746144a5-b72f-4821-bf09-d25242c818fb None admin] RBAC: Adding query filter params (name=member) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:48:38.649387 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746144a5-b72f-4821-bf09-d25242c818fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:38.650795 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746144a5-b72f-4821-bf09-d25242c818fb None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:38.658626 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 37/74] 10.222.0.39 () {56 vars in 1101 bytes} [Fri Oct 14 02:48:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:48:38.659428 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746144a5-b72f-4821-bf09-d25242c818fb None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:38.694952 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:38.709801 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cf0954-52a3-479a-9a5f-ea65d35a44d1 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:38.726567 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:38.743416 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f574c17c-b974-4649-873c-276abd0ea23b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:38.744196 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f574c17c-b974-4649-873c-276abd0ea23b None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:38.744475 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f574c17c-b974-4649-873c-276abd0ea23b None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:38.744749 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f574c17c-b974-4649-873c-276abd0ea23b None admin] PATH_INFO: `/v3/users/316b39adbf9f4b0d9c13dc4c11a24e73` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:38.746283 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f574c17c-b974-4649-873c-276abd0ea23b None admin] RBAC: Authorizing `identity:get_user(user_id=316b39adbf9f4b0d9c13dc4c11a24e73)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:38.747749 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f574c17c-b974-4649-873c-276abd0ea23b None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:38.749483 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 38/75] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Oct 14 02:48:38 2022] GET /identity/v3/users/316b39adbf9f4b0d9c13dc4c11a24e73 => generated 247 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:48:38.750223 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f574c17c-b974-4649-873c-276abd0ea23b None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:38.762024 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746144a5-b72f-4821-bf09-d25242c818fb None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:38.795705 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdfceb3-2279-4f0d-a4c6-83cedc966f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:38.796264 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdfceb3-2279-4f0d-a4c6-83cedc966f35 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:38.796544 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdfceb3-2279-4f0d-a4c6-83cedc966f35 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:38.796819 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdfceb3-2279-4f0d-a4c6-83cedc966f35 None admin] PATH_INFO: `/v3/projects/0b15574d2808402ca6940d47f10e2db8` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:38.797950 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdfceb3-2279-4f0d-a4c6-83cedc966f35 None admin] RBAC: Authorizing `identity:get_project(project_id=0b15574d2808402ca6940d47f10e2db8)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:38.799409 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdfceb3-2279-4f0d-a4c6-83cedc966f35 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:38.801040 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 38/76] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Oct 14 02:48:38 2022] GET /identity/v3/projects/0b15574d2808402ca6940d47f10e2db8 => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:48:38.806642 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdfceb3-2279-4f0d-a4c6-83cedc966f35 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:38.846815 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f574c17c-b974-4649-873c-276abd0ea23b None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:38.879971 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd0df2a-5d43-4aec-8329-464a8a894d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:38.880596 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd0df2a-5d43-4aec-8329-464a8a894d18 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:38.880876 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd0df2a-5d43-4aec-8329-464a8a894d18 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:38.881154 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd0df2a-5d43-4aec-8329-464a8a894d18 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:38.882306 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd0df2a-5d43-4aec-8329-464a8a894d18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:38.883758 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd0df2a-5d43-4aec-8329-464a8a894d18 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:38.885300 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 39/77] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:48:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:48:38.886079 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd0df2a-5d43-4aec-8329-464a8a894d18 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:38.905543 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdfceb3-2279-4f0d-a4c6-83cedc966f35 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:38.907198 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:38.938315 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094d42c0-f7a7-4471-9ff6-c00091d2d4fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:38.938949 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094d42c0-f7a7-4471-9ff6-c00091d2d4fd None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:38.939242 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094d42c0-f7a7-4471-9ff6-c00091d2d4fd None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:38.939526 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094d42c0-f7a7-4471-9ff6-c00091d2d4fd None admin] PATH_INFO: `/v3/projects/0b15574d2808402ca6940d47f10e2db8` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:38.940669 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094d42c0-f7a7-4471-9ff6-c00091d2d4fd None admin] RBAC: Authorizing `identity:get_project(project_id=0b15574d2808402ca6940d47f10e2db8)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:38.942153 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094d42c0-f7a7-4471-9ff6-c00091d2d4fd None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:38.943785 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 39/78] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Oct 14 02:48:38 2022] GET /identity/v3/projects/0b15574d2808402ca6940d47f10e2db8 => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:48:38.944541 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094d42c0-f7a7-4471-9ff6-c00091d2d4fd None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:38.953044 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:38.989073 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd0df2a-5d43-4aec-8329-464a8a894d18 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:39.025548 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab6ae46-5a46-47f1-857d-0dca580f3957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:39.026209 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab6ae46-5a46-47f1-857d-0dca580f3957 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:39.026489 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab6ae46-5a46-47f1-857d-0dca580f3957 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:39.026762 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab6ae46-5a46-47f1-857d-0dca580f3957 None admin] PATH_INFO: `/v3/users/2d06bb982e494934920981820968bc00` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:39.028415 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab6ae46-5a46-47f1-857d-0dca580f3957 None admin] RBAC: Authorizing `identity:get_user(user_id=2d06bb982e494934920981820968bc00)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:39.029907 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab6ae46-5a46-47f1-857d-0dca580f3957 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:39.031485 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 40/79] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Oct 14 02:48:38 2022] GET /identity/v3/users/2d06bb982e494934920981820968bc00 => generated 275 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:48:39.032215 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab6ae46-5a46-47f1-857d-0dca580f3957 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:39.047401 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094d42c0-f7a7-4471-9ff6-c00091d2d4fd None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:39.079564 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7c2998-340f-4637-9245-e9ec603bb703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:39.080190 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7c2998-340f-4637-9245-e9ec603bb703 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:39.080472 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7c2998-340f-4637-9245-e9ec603bb703 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:39.081038 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7c2998-340f-4637-9245-e9ec603bb703 None admin] PATH_INFO: `/v3/projects/0b15574d2808402ca6940d47f10e2db8` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:39.082171 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7c2998-340f-4637-9245-e9ec603bb703 None admin] RBAC: Authorizing `identity:get_project(project_id=0b15574d2808402ca6940d47f10e2db8)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:39.083624 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7c2998-340f-4637-9245-e9ec603bb703 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:39.085241 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 40/80] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Oct 14 02:48:38 2022] GET /identity/v3/projects/0b15574d2808402ca6940d47f10e2db8 => generated 298 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:48:39.086520 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7c2998-340f-4637-9245-e9ec603bb703 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:39.122027 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:39.135100 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab6ae46-5a46-47f1-857d-0dca580f3957 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:39.161237 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:39.169157 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1564fd-cf9d-4453-9efd-ac6b76781d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:39.169831 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1564fd-cf9d-4453-9efd-ac6b76781d7a None admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:39.170116 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1564fd-cf9d-4453-9efd-ac6b76781d7a None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:39.170391 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1564fd-cf9d-4453-9efd-ac6b76781d7a None admin] PATH_INFO: `/v3/projects/0b15574d2808402ca6940d47f10e2db8/users/2d06bb982e494934920981820968bc00/roles/9bd0470b544240209a420f640c79c06c` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:39.173806 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1564fd-cf9d-4453-9efd-ac6b76781d7a None admin] RBAC: Authorizing `identity:create_grant(project_id=0b15574d2808402ca6940d47f10e2db8, user_id=2d06bb982e494934920981820968bc00, role_id=9bd0470b544240209a420f640c79c06c)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:39.173806 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1564fd-cf9d-4453-9efd-ac6b76781d7a None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:39.184894 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7c2998-340f-4637-9245-e9ec603bb703 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:39.218191 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5050b544-8f57-4975-8a16-cbbda420e2c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:39.218791 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5050b544-8f57-4975-8a16-cbbda420e2c5 None admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:39.219109 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5050b544-8f57-4975-8a16-cbbda420e2c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:39.219385 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5050b544-8f57-4975-8a16-cbbda420e2c5 None admin] PATH_INFO: `/v3/users` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:39.220024 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5050b544-8f57-4975-8a16-cbbda420e2c5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:39.221439 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5050b544-8f57-4975-8a16-cbbda420e2c5 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:39.247951 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 41/81] 10.222.0.39 () {58 vars in 1439 bytes} [Fri Oct 14 02:48:39 2022] PUT /identity/v3/projects/0b15574d2808402ca6940d47f10e2db8/users/2d06bb982e494934920981820968bc00/roles/9bd0470b544240209a420f640c79c06c => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:48:39.248790 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1564fd-cf9d-4453-9efd-ac6b76781d7a None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:39.349795 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 41/82] 10.222.0.39 () {60 vars in 1132 bytes} [Fri Oct 14 02:48:39 2022] POST /identity/v3/users => generated 289 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:48:39.349795 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5050b544-8f57-4975-8a16-cbbda420e2c5 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:39.366393 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1564fd-cf9d-4453-9efd-ac6b76781d7a None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:39.408364 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba6ae69-5a8d-4d44-90fc-2cd6a14be09c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:39.409085 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba6ae69-5a8d-4d44-90fc-2cd6a14be09c None admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:39.409399 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba6ae69-5a8d-4d44-90fc-2cd6a14be09c None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:39.409684 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba6ae69-5a8d-4d44-90fc-2cd6a14be09c None admin] PATH_INFO: `/v3/projects/0b15574d2808402ca6940d47f10e2db8/users/2d06bb982e494934920981820968bc00/roles/d695dc72739e4aa7a50d5fb73ad1a91f` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:39.412181 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba6ae69-5a8d-4d44-90fc-2cd6a14be09c None admin] RBAC: Authorizing `identity:create_grant(project_id=0b15574d2808402ca6940d47f10e2db8, user_id=2d06bb982e494934920981820968bc00, role_id=d695dc72739e4aa7a50d5fb73ad1a91f)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:39.413799 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba6ae69-5a8d-4d44-90fc-2cd6a14be09c None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:39.483865 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5050b544-8f57-4975-8a16-cbbda420e2c5 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:39.519459 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 42/83] 10.222.0.39 () {58 vars in 1439 bytes} [Fri Oct 14 02:48:39 2022] PUT /identity/v3/projects/0b15574d2808402ca6940d47f10e2db8/users/2d06bb982e494934920981820968bc00/roles/d695dc72739e4aa7a50d5fb73ad1a91f => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:48:39.521297 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba6ae69-5a8d-4d44-90fc-2cd6a14be09c None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:39.525213 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:39.527556 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3752a9b5-6ab6-4f8c-b20f-0a38bb73a993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:39.529358 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3752a9b5-6ab6-4f8c-b20f-0a38bb73a993 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:39.529647 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3752a9b5-6ab6-4f8c-b20f-0a38bb73a993 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:39.530062 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3752a9b5-6ab6-4f8c-b20f-0a38bb73a993 None admin] PATH_INFO: `/v3/projects/ac970cbd2d634d1d9fa09ffd8070dc34` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:39.531368 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3752a9b5-6ab6-4f8c-b20f-0a38bb73a993 None admin] RBAC: Authorizing `identity:get_project(project_id=ac970cbd2d634d1d9fa09ffd8070dc34)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:39.533014 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3752a9b5-6ab6-4f8c-b20f-0a38bb73a993 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:39.535110 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 42/84] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Oct 14 02:48:39 2022] GET /identity/v3/projects/ac970cbd2d634d1d9fa09ffd8070dc34 => generated 312 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:48:39.539912 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:39.541800 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3752a9b5-6ab6-4f8c-b20f-0a38bb73a993 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:39.678574 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba6ae69-5a8d-4d44-90fc-2cd6a14be09c None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:39.688285 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3752a9b5-6ab6-4f8c-b20f-0a38bb73a993 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:39.734517 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e56ae9-bb2f-4250-a182-a4e97f3b9d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:39.735394 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e56ae9-bb2f-4250-a182-a4e97f3b9d9f None admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:39.735781 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e56ae9-bb2f-4250-a182-a4e97f3b9d9f None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:39.736151 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e56ae9-bb2f-4250-a182-a4e97f3b9d9f None admin] PATH_INFO: `/v3/projects/0b15574d2808402ca6940d47f10e2db8/users/316b39adbf9f4b0d9c13dc4c11a24e73/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:39.738650 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e56ae9-bb2f-4250-a182-a4e97f3b9d9f None admin] RBAC: Authorizing `identity:create_grant(project_id=0b15574d2808402ca6940d47f10e2db8, user_id=316b39adbf9f4b0d9c13dc4c11a24e73, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:39.740327 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e56ae9-bb2f-4250-a182-a4e97f3b9d9f None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:39.740602 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9fa6d8-8acd-4fee-8cd4-941105373f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:39.741413 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9fa6d8-8acd-4fee-8cd4-941105373f14 None admin] REQUEST_METHOD: `PUT` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:39.741898 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9fa6d8-8acd-4fee-8cd4-941105373f14 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:39.742383 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9fa6d8-8acd-4fee-8cd4-941105373f14 None admin] PATH_INFO: `/v3/projects/ac970cbd2d634d1d9fa09ffd8070dc34/users/2d06bb982e494934920981820968bc00/roles/9bd0470b544240209a420f640c79c06c` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:39.744490 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9fa6d8-8acd-4fee-8cd4-941105373f14 None admin] RBAC: Authorizing `identity:create_grant(project_id=ac970cbd2d634d1d9fa09ffd8070dc34, user_id=2d06bb982e494934920981820968bc00, role_id=9bd0470b544240209a420f640c79c06c)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:39.746461 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9fa6d8-8acd-4fee-8cd4-941105373f14 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:39.815428 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 43/85] 10.222.0.39 () {58 vars in 1439 bytes} [Fri Oct 14 02:48:39 2022] PUT /identity/v3/projects/0b15574d2808402ca6940d47f10e2db8/users/316b39adbf9f4b0d9c13dc4c11a24e73/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:48:39.857212 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 43/86] 10.222.0.39 () {58 vars in 1439 bytes} [Fri Oct 14 02:48:39 2022] PUT /identity/v3/projects/ac970cbd2d634d1d9fa09ffd8070dc34/users/2d06bb982e494934920981820968bc00/roles/9bd0470b544240209a420f640c79c06c => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 02:48:41.861109 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e57d4c-8e18-401f-b9b9-b43a27f1e6a7 None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:41.861614 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e57d4c-8e18-401f-b9b9-b43a27f1e6a7 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:41.861685 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e57d4c-8e18-401f-b9b9-b43a27f1e6a7 None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:41.863094 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 44/87] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:48:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:48:41.868588 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e73306-32a0-4fb3-87ba-537c751032b0 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:41.869367 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e73306-32a0-4fb3-87ba-537c751032b0 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:41.869854 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e73306-32a0-4fb3-87ba-537c751032b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:41.894809 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-13e73306-32a0-4fb3-87ba-537c751032b0 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:48:41.896043 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-13e73306-32a0-4fb3-87ba-537c751032b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:48:41.965319 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:41.988198 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 44/88] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:48:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:48:42.531233 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e57d4c-8e18-401f-b9b9-b43a27f1e6a7 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:42.612153 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:42.631809 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efde47f-af7a-4088-9f95-998a2a529f87 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:42.632145 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efde47f-af7a-4088-9f95-998a2a529f87 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:42.632469 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efde47f-af7a-4088-9f95-998a2a529f87 None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:42.635568 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 45/89] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:48:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:48:42.650705 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9702c4ad-6ea7-417e-a1c9-b9eaa2750386 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:42.652153 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e57d4c-8e18-401f-b9b9-b43a27f1e6a7 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:42.653498 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9702c4ad-6ea7-417e-a1c9-b9eaa2750386 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:42.653992 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9702c4ad-6ea7-417e-a1c9-b9eaa2750386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:42.690046 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-9702c4ad-6ea7-417e-a1c9-b9eaa2750386 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:48:42.691386 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-9702c4ad-6ea7-417e-a1c9-b9eaa2750386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:48:42.702101 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab5fab8-b5db-4a88-b34d-9a17a99f07a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:42.703031 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab5fab8-b5db-4a88-b34d-9a17a99f07a7 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:42.703522 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab5fab8-b5db-4a88-b34d-9a17a99f07a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:42.703986 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab5fab8-b5db-4a88-b34d-9a17a99f07a7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:42.722843 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab5fab8-b5db-4a88-b34d-9a17a99f07a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:48:42.735988 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-8ab5fab8-b5db-4a88-b34d-9a17a99f07a7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 14 02:48:42.735988 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:48:42.735988 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:48:42.735988 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:48:42.735988 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:48:42.735988 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:48:42.735988 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:48:42.735988 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:48:42.735988 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:48:42.735988 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:48:42.735988 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:48:42.735988 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:48:42.735988 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:48:42.735988 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:48:42.735988 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:48:42.735988 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:48:42.736842 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:48:42.736842 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:48:42.736842 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:48:42.736842 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:48:42.736842 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:48:42.736842 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:48:42.736842 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:48:42.736842 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:48:42.736842 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:48:42.736842 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:48:42.736842 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:48:42.736842 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:48:42.736842 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:48:42.736842 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:48:42.736842 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:48:42.736842 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:48:42.737599 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:48:42.737599 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:48:42.737599 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:48:42.737599 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:48:42.737599 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:48:42.737599 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:48:42.737599 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:48:42.737599 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:48:42.737599 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:48:42.737599 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:48:42.737599 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 14 02:48:42.737599 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:48:42.743290 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 45/90] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Oct 14 02:48:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:48:42.747427 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab5fab8-b5db-4a88-b34d-9a17a99f07a7 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:42.863123 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 46/91] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:48:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 14 02:48:42.869457 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7c8600-4c5a-432b-8696-d29c45656520 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:42.870048 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7c8600-4c5a-432b-8696-d29c45656520 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:42.870048 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7c8600-4c5a-432b-8696-d29c45656520 None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:42.871663 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 47/92] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:48:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:48:42.873163 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab5fab8-b5db-4a88-b34d-9a17a99f07a7 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:42.876670 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb89ecf-8feb-4868-bc0a-11ace442625e None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:42.876670 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb89ecf-8feb-4868-bc0a-11ace442625e None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:42.877007 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb89ecf-8feb-4868-bc0a-11ace442625e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:42.913612 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb76426-da7a-42b0-8eb2-9267007cdae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:42.914592 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb76426-da7a-42b0-8eb2-9267007cdae6 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:42.915070 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb76426-da7a-42b0-8eb2-9267007cdae6 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:42.915673 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb76426-da7a-42b0-8eb2-9267007cdae6 None admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:42.916221 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-edb89ecf-8feb-4868-bc0a-11ace442625e None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:48:42.916770 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb76426-da7a-42b0-8eb2-9267007cdae6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:48:42.917272 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb76426-da7a-42b0-8eb2-9267007cdae6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:42.917437 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-edb89ecf-8feb-4868-bc0a-11ace442625e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:48:42.925779 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:42.926317 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb76426-da7a-42b0-8eb2-9267007cdae6 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:42.941784 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 46/93] 10.222.0.39 () {56 vars in 1101 bytes} [Fri Oct 14 02:48:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:48:42.943578 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb76426-da7a-42b0-8eb2-9267007cdae6 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:43.080009 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb76426-da7a-42b0-8eb2-9267007cdae6 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:43.086795 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 48/94] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:48:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:48:43.088081 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fd3563-c9f1-43b2-9ddc-5bf63b7b8401 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:43.088368 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fd3563-c9f1-43b2-9ddc-5bf63b7b8401 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:43.088646 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fd3563-c9f1-43b2-9ddc-5bf63b7b8401 None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:43.098217 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 49/95] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:48:43 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:48:43.103449 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa132332-71b8-4f1c-9639-6a50777c9628 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:43.103738 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa132332-71b8-4f1c-9639-6a50777c9628 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:43.104018 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa132332-71b8-4f1c-9639-6a50777c9628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:43.118497 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5f44ec-90a2-4dc4-a6b0-cd40bad1592c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:43.119465 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5f44ec-90a2-4dc4-a6b0-cd40bad1592c None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:43.119959 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5f44ec-90a2-4dc4-a6b0-cd40bad1592c None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:43.120467 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5f44ec-90a2-4dc4-a6b0-cd40bad1592c None admin] PATH_INFO: `/v3/users/2d06bb982e494934920981820968bc00` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:43.122162 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5f44ec-90a2-4dc4-a6b0-cd40bad1592c None admin] RBAC: Authorizing `identity:get_user(user_id=2d06bb982e494934920981820968bc00)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:43.124061 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5f44ec-90a2-4dc4-a6b0-cd40bad1592c None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:43.126033 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 47/96] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Oct 14 02:48:42 2022] GET /identity/v3/users/2d06bb982e494934920981820968bc00 => generated 275 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:48:43.131635 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5f44ec-90a2-4dc4-a6b0-cd40bad1592c None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:43.141510 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-fa132332-71b8-4f1c-9639-6a50777c9628 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:48:43.142732 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-fa132332-71b8-4f1c-9639-6a50777c9628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:48:43.190084 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:43.197891 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:43.215994 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 50/97] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:48:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 14 02:48:43.217608 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fd86c5-a931-4f87-b683-27391ed98955 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:43.218126 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fd86c5-a931-4f87-b683-27391ed98955 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:43.218607 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fd86c5-a931-4f87-b683-27391ed98955 None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:43.220182 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 51/98] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:48:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:48:43.221085 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fd86c5-a931-4f87-b683-27391ed98955 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:43.253627 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5f44ec-90a2-4dc4-a6b0-cd40bad1592c None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:43.298581 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810447ad-7f42-400d-85cd-ef48b6124f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:43.299203 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810447ad-7f42-400d-85cd-ef48b6124f81 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:43.299490 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810447ad-7f42-400d-85cd-ef48b6124f81 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:43.299772 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810447ad-7f42-400d-85cd-ef48b6124f81 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:43.330243 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fd86c5-a931-4f87-b683-27391ed98955 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:43.373942 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3ea947-462d-43a2-a091-a4fe418967cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:43.374509 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3ea947-462d-43a2-a091-a4fe418967cc None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:43.374928 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3ea947-462d-43a2-a091-a4fe418967cc None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:43.375342 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3ea947-462d-43a2-a091-a4fe418967cc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:43.391275 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3ea947-462d-43a2-a091-a4fe418967cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:48:43.394143 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810447ad-7f42-400d-85cd-ef48b6124f81 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:43.395626 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810447ad-7f42-400d-85cd-ef48b6124f81 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:43.397115 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-cf3ea947-462d-43a2-a091-a4fe418967cc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 14 02:48:43.397115 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:48:43.397115 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:48:43.397115 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:48:43.397115 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:48:43.397115 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:48:43.397115 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:48:43.397115 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:48:43.397115 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:48:43.397115 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:48:43.397115 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:48:43.397115 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:48:43.397115 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:48:43.397115 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:48:43.397115 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:48:43.397115 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:48:43.398059 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:48:43.398059 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:48:43.398059 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:48:43.398059 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:48:43.398059 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:48:43.398059 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:48:43.398059 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:48:43.398059 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:48:43.398059 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:48:43.398059 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:48:43.398059 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:48:43.398059 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:48:43.398059 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:48:43.398059 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:48:43.398059 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:48:43.398059 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:48:43.398839 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:48:43.398839 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:48:43.398839 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:48:43.398839 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:48:43.398839 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:48:43.398839 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:48:43.398839 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:48:43.398839 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:48:43.398839 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:48:43.398839 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:48:43.398839 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 14 02:48:43.398839 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:48:43.404040 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 52/99] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Oct 14 02:48:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:48:43.407233 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815f029f-1f82-48ab-bbcd-217d165e198e None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:43.407938 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815f029f-1f82-48ab-bbcd-217d165e198e None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:43.408234 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815f029f-1f82-48ab-bbcd-217d165e198e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:43.434916 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-815f029f-1f82-48ab-bbcd-217d165e198e None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:48:43.437448 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-815f029f-1f82-48ab-bbcd-217d165e198e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:48:43.464446 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:43.480146 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:43.509854 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 48/100] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Oct 14 02:48:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 14 02:48:43.511142 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810447ad-7f42-400d-85cd-ef48b6124f81 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:43.538043 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 53/101] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:48:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:48:43.540072 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815f029f-1f82-48ab-bbcd-217d165e198e None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:43.623201 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810447ad-7f42-400d-85cd-ef48b6124f81 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:43.644167 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815f029f-1f82-48ab-bbcd-217d165e198e None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:43.657230 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc5931e-8285-40ef-8558-8c75f8f07889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:43.657900 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc5931e-8285-40ef-8558-8c75f8f07889 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:43.658188 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc5931e-8285-40ef-8558-8c75f8f07889 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:43.658465 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc5931e-8285-40ef-8558-8c75f8f07889 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:43.669677 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc5931e-8285-40ef-8558-8c75f8f07889 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:48:43.674503 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-ffc5931e-8285-40ef-8558-8c75f8f07889 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 14 02:48:43.674503 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:48:43.674503 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:48:43.674503 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:48:43.674503 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:48:43.674503 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:48:43.674503 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:48:43.674503 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:48:43.674503 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:48:43.674503 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:48:43.674503 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:48:43.674503 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:48:43.674503 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:48:43.674503 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:48:43.674503 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:48:43.674503 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:48:43.675526 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:48:43.675526 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:48:43.675526 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:48:43.675526 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:48:43.675526 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:48:43.675526 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:48:43.675526 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:48:43.675526 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:48:43.675526 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:48:43.675526 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:48:43.675526 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:48:43.675526 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:48:43.675526 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:48:43.675526 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:48:43.675526 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:48:43.675526 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:48:43.676478 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:48:43.676478 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:48:43.676478 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:48:43.676478 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:48:43.676478 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:48:43.676478 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:48:43.676478 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:48:43.676478 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:48:43.676478 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:48:43.676478 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:48:43.676478 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 14 02:48:43.676478 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:48:43.680506 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 49/102] 10.222.0.39 () {56 vars in 1114 bytes} [Fri Oct 14 02:48:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:48:43.681349 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc5931e-8285-40ef-8558-8c75f8f07889 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:43.683337 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf1152e-6589-4cb4-8772-0222be05e45c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:43.684253 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf1152e-6589-4cb4-8772-0222be05e45c None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:43.684682 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf1152e-6589-4cb4-8772-0222be05e45c None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:43.685217 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf1152e-6589-4cb4-8772-0222be05e45c None admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:43.685927 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf1152e-6589-4cb4-8772-0222be05e45c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:48:43.686243 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf1152e-6589-4cb4-8772-0222be05e45c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:43.687647 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf1152e-6589-4cb4-8772-0222be05e45c None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:43.705155 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 54/103] 10.222.0.39 () {56 vars in 1101 bytes} [Fri Oct 14 02:48:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:48:43.705155 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7072b673-5afd-4c10-b5f7-47531445a7b7 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:43.705155 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7072b673-5afd-4c10-b5f7-47531445a7b7 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:43.705155 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7072b673-5afd-4c10-b5f7-47531445a7b7 None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:43.705155 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 55/104] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:48:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:48:43.705155 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7072b673-5afd-4c10-b5f7-47531445a7b7 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:43.709879 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:43.737194 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:43.803190 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc5931e-8285-40ef-8558-8c75f8f07889 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:43.809224 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7072b673-5afd-4c10-b5f7-47531445a7b7 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:43.837502 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726ad006-a5f1-4f53-bf7b-e28d7c84e4b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:43.838155 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726ad006-a5f1-4f53-bf7b-e28d7c84e4b8 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:43.838440 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726ad006-a5f1-4f53-bf7b-e28d7c84e4b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:43.838717 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726ad006-a5f1-4f53-bf7b-e28d7c84e4b8 None admin] PATH_INFO: `/v3/projects/0b15574d2808402ca6940d47f10e2db8` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:43.839822 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726ad006-a5f1-4f53-bf7b-e28d7c84e4b8 None admin] RBAC: Authorizing `identity:get_project(project_id=0b15574d2808402ca6940d47f10e2db8)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:43.841263 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726ad006-a5f1-4f53-bf7b-e28d7c84e4b8 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:43.842912 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 50/105] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Oct 14 02:48:43 2022] GET /identity/v3/projects/0b15574d2808402ca6940d47f10e2db8 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:48:43.843247 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0d07eb-94c4-4d1f-9a10-c3384cb7b494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:43.843672 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726ad006-a5f1-4f53-bf7b-e28d7c84e4b8 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:43.843876 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0d07eb-94c4-4d1f-9a10-c3384cb7b494 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:43.844165 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0d07eb-94c4-4d1f-9a10-c3384cb7b494 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:43.844440 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0d07eb-94c4-4d1f-9a10-c3384cb7b494 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:43.855014 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0d07eb-94c4-4d1f-9a10-c3384cb7b494 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:48:43.859759 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-7f0d07eb-94c4-4d1f-9a10-c3384cb7b494 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:48:43.859759 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:48:43.859759 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:48:43.859759 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:48:43.859759 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:48:43.859759 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:48:43.859759 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:48:43.859759 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:48:43.859759 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:48:43.859759 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:48:43.859759 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:48:43.859759 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:48:43.859759 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:48:43.859759 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:48:43.859759 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:48:43.859759 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:48:43.860588 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:48:43.860588 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:48:43.860588 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:48:43.860588 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:48:43.860588 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:48:43.860588 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:48:43.860588 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:48:43.860588 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:48:43.860588 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:48:43.860588 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:48:43.860588 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:48:43.860588 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:48:43.860588 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:48:43.860588 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:48:43.860588 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:48:43.860588 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:48:43.861345 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:48:43.861345 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:48:43.861345 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:48:43.861345 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:48:43.861345 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:48:43.861345 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:48:43.861345 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:48:43.861345 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:48:43.861345 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:48:43.861345 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:48:43.861345 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:48:43.861345 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:48:43.861345 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 56/106] 10.222.0.39 () {56 vars in 1096 bytes} [Fri Oct 14 02:48:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:48:43.862426 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978d5a53-1b94-4220-916d-9bb25f552814 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:43.862426 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978d5a53-1b94-4220-916d-9bb25f552814 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:43.862567 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978d5a53-1b94-4220-916d-9bb25f552814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:43.879373 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-978d5a53-1b94-4220-916d-9bb25f552814 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:48:43.880563 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-978d5a53-1b94-4220-916d-9bb25f552814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:48:43.928034 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:43.942193 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726ad006-a5f1-4f53-bf7b-e28d7c84e4b8 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:43.944072 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 57/107] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:48:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:48:43.945411 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978d5a53-1b94-4220-916d-9bb25f552814 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:43.976678 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:43.982792 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1828a423-5e4f-4572-8f4f-ad9beee14dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:43.986089 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1828a423-5e4f-4572-8f4f-ad9beee14dcd None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:43.986089 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1828a423-5e4f-4572-8f4f-ad9beee14dcd None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:43.986089 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1828a423-5e4f-4572-8f4f-ad9beee14dcd None admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:43.986089 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1828a423-5e4f-4572-8f4f-ad9beee14dcd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:48:43.986089 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1828a423-5e4f-4572-8f4f-ad9beee14dcd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:43.986469 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1828a423-5e4f-4572-8f4f-ad9beee14dcd None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:43.994788 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 51/108] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Oct 14 02:48:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:48:43.995619 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1828a423-5e4f-4572-8f4f-ad9beee14dcd None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:44.066144 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978d5a53-1b94-4220-916d-9bb25f552814 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:44.101778 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe284d3-2278-4d4b-ac96-a5ce087654e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:44.102430 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe284d3-2278-4d4b-ac96-a5ce087654e3 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:44.102714 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe284d3-2278-4d4b-ac96-a5ce087654e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:44.102987 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe284d3-2278-4d4b-ac96-a5ce087654e3 None admin] PATH_INFO: `/v3/users/e4d39d44cb9e4196ada8e0e4e4e5a43f` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:44.103517 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1828a423-5e4f-4572-8f4f-ad9beee14dcd None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:44.120048 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe284d3-2278-4d4b-ac96-a5ce087654e3 None admin] RBAC: Authorizing `identity:get_user(user_id=e4d39d44cb9e4196ada8e0e4e4e5a43f)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:44.121520 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe284d3-2278-4d4b-ac96-a5ce087654e3 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:44.127560 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 58/109] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Oct 14 02:48:43 2022] GET /identity/v3/users/e4d39d44cb9e4196ada8e0e4e4e5a43f => generated 289 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:48:44.127560 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe284d3-2278-4d4b-ac96-a5ce087654e3 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:44.144445 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdcef14-16ad-4deb-b459-c7bb3f45efac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:44.145196 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdcef14-16ad-4deb-b459-c7bb3f45efac None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:44.145481 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdcef14-16ad-4deb-b459-c7bb3f45efac None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:44.145779 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdcef14-16ad-4deb-b459-c7bb3f45efac None admin] PATH_INFO: `/v3/domains/default` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:44.146891 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdcef14-16ad-4deb-b459-c7bb3f45efac None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:44.148374 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdcef14-16ad-4deb-b459-c7bb3f45efac None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:44.149958 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 52/110] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:48:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:48:44.149958 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdcef14-16ad-4deb-b459-c7bb3f45efac None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:44.168241 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:44.211695 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:44.233624 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe284d3-2278-4d4b-ac96-a5ce087654e3 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:44.252811 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdcef14-16ad-4deb-b459-c7bb3f45efac None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:44.267941 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4d2db3-b711-4a08-9209-753a871751a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:44.268549 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d2db3-b711-4a08-9209-753a871751a0 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:44.268834 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d2db3-b711-4a08-9209-753a871751a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:44.269109 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d2db3-b711-4a08-9209-753a871751a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:44.269934 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4d2db3-b711-4a08-9209-753a871751a0 None admin] RBAC: Adding query filter params (role.id=9bd0470b544240209a420f640c79c06c, scope.project.id=0b15574d2808402ca6940d47f10e2db8, user.id=2d06bb982e494934920981820968bc00) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:48:44.270252 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4d2db3-b711-4a08-9209-753a871751a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:44.271712 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4d2db3-b711-4a08-9209-753a871751a0 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:44.277428 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 59/111] 10.222.0.39 () {56 vars in 1374 bytes} [Fri Oct 14 02:48:44 2022] GET /identity/v3/role_assignments?user.id=2d06bb982e494934920981820968bc00&scope.project.id=0b15574d2808402ca6940d47f10e2db8&role.id=9bd0470b544240209a420f640c79c06c => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:48:44.278247 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4d2db3-b711-4a08-9209-753a871751a0 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:44.285445 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985612ba-e748-4438-9bc2-3e34a7c90c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:44.286105 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985612ba-e748-4438-9bc2-3e34a7c90c3d None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:44.286394 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985612ba-e748-4438-9bc2-3e34a7c90c3d None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:44.286668 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985612ba-e748-4438-9bc2-3e34a7c90c3d None admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:44.287363 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985612ba-e748-4438-9bc2-3e34a7c90c3d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:48:44.287673 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985612ba-e748-4438-9bc2-3e34a7c90c3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:44.289153 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985612ba-e748-4438-9bc2-3e34a7c90c3d None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:44.298758 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 53/112] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Oct 14 02:48:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:48:44.299426 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985612ba-e748-4438-9bc2-3e34a7c90c3d None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:44.388400 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:44.391032 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4d2db3-b711-4a08-9209-753a871751a0 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:44.411984 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985612ba-e748-4438-9bc2-3e34a7c90c3d None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:44.428209 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42a4768-4df2-4ced-98f1-d384404538c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:44.428864 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42a4768-4df2-4ced-98f1-d384404538c6 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:44.429710 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42a4768-4df2-4ced-98f1-d384404538c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:44.430098 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42a4768-4df2-4ced-98f1-d384404538c6 None admin] PATH_INFO: `/v3/users/2d06bb982e494934920981820968bc00` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:44.431216 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42a4768-4df2-4ced-98f1-d384404538c6 None admin] RBAC: Authorizing `identity:get_user(user_id=2d06bb982e494934920981820968bc00)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:44.432778 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42a4768-4df2-4ced-98f1-d384404538c6 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:44.439309 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 60/113] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Oct 14 02:48:44 2022] GET /identity/v3/users/2d06bb982e494934920981820968bc00 => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:48:44.441142 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42a4768-4df2-4ced-98f1-d384404538c6 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:44.453000 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed7d1d3-e137-49f9-9c15-4be8db878dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:44.453651 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed7d1d3-e137-49f9-9c15-4be8db878dd6 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:44.453964 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed7d1d3-e137-49f9-9c15-4be8db878dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:44.454248 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed7d1d3-e137-49f9-9c15-4be8db878dd6 None admin] PATH_INFO: `/v3/projects/0b15574d2808402ca6940d47f10e2db8` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:44.455427 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed7d1d3-e137-49f9-9c15-4be8db878dd6 None admin] RBAC: Authorizing `identity:get_project(project_id=0b15574d2808402ca6940d47f10e2db8)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:44.456195 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:44.457284 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed7d1d3-e137-49f9-9c15-4be8db878dd6 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:44.458754 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 54/114] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Oct 14 02:48:44 2022] GET /identity/v3/projects/0b15574d2808402ca6940d47f10e2db8 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:48:44.459548 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed7d1d3-e137-49f9-9c15-4be8db878dd6 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:44.553188 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42a4768-4df2-4ced-98f1-d384404538c6 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:44.572048 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed7d1d3-e137-49f9-9c15-4be8db878dd6 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:44.585632 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826a2334-8819-48a8-8fbd-b90378f838b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:44.586267 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826a2334-8819-48a8-8fbd-b90378f838b3 None admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:44.586551 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826a2334-8819-48a8-8fbd-b90378f838b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:44.586833 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826a2334-8819-48a8-8fbd-b90378f838b3 None admin] PATH_INFO: `/v3/projects` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:44.587483 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826a2334-8819-48a8-8fbd-b90378f838b3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:44.588934 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826a2334-8819-48a8-8fbd-b90378f838b3 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:44.604686 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d541cd95-d9ae-418c-bdcc-d64a76fe95d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:44.605317 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d541cd95-d9ae-418c-bdcc-d64a76fe95d0 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:44.605607 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d541cd95-d9ae-418c-bdcc-d64a76fe95d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:44.605906 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d541cd95-d9ae-418c-bdcc-d64a76fe95d0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:44.615699 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d541cd95-d9ae-418c-bdcc-d64a76fe95d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:48:44.622293 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-d541cd95-d9ae-418c-bdcc-d64a76fe95d0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 14 02:48:44.622293 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:48:44.622293 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:48:44.622293 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:48:44.622293 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:48:44.622293 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:48:44.622293 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:48:44.622293 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:48:44.622293 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:48:44.622293 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:48:44.622293 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:48:44.622293 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:48:44.622293 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:48:44.622293 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:48:44.622293 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:48:44.622293 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:48:44.623306 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:48:44.623306 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:48:44.623306 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:48:44.623306 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:48:44.623306 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:48:44.623306 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:48:44.623306 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:48:44.623306 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:48:44.623306 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:48:44.623306 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:48:44.623306 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:48:44.623306 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:48:44.623306 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:48:44.623306 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:48:44.623306 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:48:44.623306 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:48:44.624102 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:48:44.624102 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:48:44.624102 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:48:44.624102 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:48:44.624102 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:48:44.624102 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:48:44.624102 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:48:44.624102 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:48:44.624102 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:48:44.624102 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:48:44.624102 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 14 02:48:44.624102 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:48:44.624102 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 55/115] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Oct 14 02:48:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:48:44.624735 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d541cd95-d9ae-418c-bdcc-d64a76fe95d0 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:44.632183 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:44.671931 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 61/116] 10.222.0.39 () {60 vars in 1141 bytes} [Fri Oct 14 02:48:44 2022] POST /identity/v3/projects => generated 302 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:48:44.673087 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826a2334-8819-48a8-8fbd-b90378f838b3 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:44.739632 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d541cd95-d9ae-418c-bdcc-d64a76fe95d0 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:44.753528 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:44.777273 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ada08ed-f054-4923-a472-bfab728039b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:44.777962 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ada08ed-f054-4923-a472-bfab728039b6 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:44.778251 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ada08ed-f054-4923-a472-bfab728039b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:44.778534 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ada08ed-f054-4923-a472-bfab728039b6 None admin] PATH_INFO: `/v3/users/316b39adbf9f4b0d9c13dc4c11a24e73` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:44.779719 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ada08ed-f054-4923-a472-bfab728039b6 None admin] RBAC: Authorizing `identity:get_user(user_id=316b39adbf9f4b0d9c13dc4c11a24e73)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:44.781233 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ada08ed-f054-4923-a472-bfab728039b6 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:44.782909 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 56/117] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Oct 14 02:48:44 2022] GET /identity/v3/users/316b39adbf9f4b0d9c13dc4c11a24e73 => generated 247 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:48:44.782909 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ada08ed-f054-4923-a472-bfab728039b6 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:44.803839 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826a2334-8819-48a8-8fbd-b90378f838b3 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:44.840655 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e94d0c-0d35-4dac-8ee4-9118fc4e0420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:44.841278 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e94d0c-0d35-4dac-8ee4-9118fc4e0420 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:44.841567 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e94d0c-0d35-4dac-8ee4-9118fc4e0420 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:44.841885 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e94d0c-0d35-4dac-8ee4-9118fc4e0420 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:44.876844 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:44.904693 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ada08ed-f054-4923-a472-bfab728039b6 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:44.916939 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e94d0c-0d35-4dac-8ee4-9118fc4e0420 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:44.918465 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e94d0c-0d35-4dac-8ee4-9118fc4e0420 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:44.941786 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c82b9c-6ea4-4615-9e68-b792e733e288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:44.942775 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c82b9c-6ea4-4615-9e68-b792e733e288 None admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:44.943369 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c82b9c-6ea4-4615-9e68-b792e733e288 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:44.943660 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c82b9c-6ea4-4615-9e68-b792e733e288 None admin] PATH_INFO: `/v3/projects/0b15574d2808402ca6940d47f10e2db8/users/e4d39d44cb9e4196ada8e0e4e4e5a43f/roles/3026af19537b46cab1318d0e13ae4163` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:44.945730 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c82b9c-6ea4-4615-9e68-b792e733e288 None admin] RBAC: Authorizing `identity:create_grant(project_id=0b15574d2808402ca6940d47f10e2db8, user_id=e4d39d44cb9e4196ada8e0e4e4e5a43f, role_id=3026af19537b46cab1318d0e13ae4163)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:44.947288 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c82b9c-6ea4-4615-9e68-b792e733e288 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:44.979160 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:45.015574 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 57/118] 10.222.0.39 () {58 vars in 1439 bytes} [Fri Oct 14 02:48:44 2022] PUT /identity/v3/projects/0b15574d2808402ca6940d47f10e2db8/users/e4d39d44cb9e4196ada8e0e4e4e5a43f/roles/3026af19537b46cab1318d0e13ae4163 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:48:45.016526 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c82b9c-6ea4-4615-9e68-b792e733e288 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:45.025520 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 62/119] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Oct 14 02:48:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 14 02:48:45.025520 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e94d0c-0d35-4dac-8ee4-9118fc4e0420 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:45.149161 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c82b9c-6ea4-4615-9e68-b792e733e288 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:45.164066 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e94d0c-0d35-4dac-8ee4-9118fc4e0420 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:45.194257 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3e1442-525d-42ee-96ef-f30e21ce6d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:45.197541 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e1442-525d-42ee-96ef-f30e21ce6d09 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:45.197541 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e1442-525d-42ee-96ef-f30e21ce6d09 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:45.197541 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e1442-525d-42ee-96ef-f30e21ce6d09 None admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:45.197541 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3e1442-525d-42ee-96ef-f30e21ce6d09 None admin] RBAC: Adding query filter params (name=member) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:48:45.197825 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3e1442-525d-42ee-96ef-f30e21ce6d09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:45.200223 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3e1442-525d-42ee-96ef-f30e21ce6d09 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:45.201429 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:45.211531 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa012c29-8068-41b9-a3f7-ad3c5868be04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:45.211701 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 58/120] 10.222.0.39 () {56 vars in 1101 bytes} [Fri Oct 14 02:48:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:48:45.212142 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa012c29-8068-41b9-a3f7-ad3c5868be04 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:45.212466 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa012c29-8068-41b9-a3f7-ad3c5868be04 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:45.212566 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3e1442-525d-42ee-96ef-f30e21ce6d09 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:45.212796 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa012c29-8068-41b9-a3f7-ad3c5868be04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:45.248445 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:45.295058 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa012c29-8068-41b9-a3f7-ad3c5868be04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:45.297353 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa012c29-8068-41b9-a3f7-ad3c5868be04 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:45.344128 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3e1442-525d-42ee-96ef-f30e21ce6d09 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:45.387034 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd5e908-2a61-457c-92ac-a6fd2f0fd38e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:45.387806 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd5e908-2a61-457c-92ac-a6fd2f0fd38e None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:45.388160 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd5e908-2a61-457c-92ac-a6fd2f0fd38e None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:45.388510 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd5e908-2a61-457c-92ac-a6fd2f0fd38e None admin] PATH_INFO: `/v3/projects/0b15574d2808402ca6940d47f10e2db8` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:45.389732 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd5e908-2a61-457c-92ac-a6fd2f0fd38e None admin] RBAC: Authorizing `identity:get_project(project_id=0b15574d2808402ca6940d47f10e2db8)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:45.391267 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd5e908-2a61-457c-92ac-a6fd2f0fd38e None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:45.392996 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 59/121] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Oct 14 02:48:45 2022] GET /identity/v3/projects/0b15574d2808402ca6940d47f10e2db8 => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:48:45.393936 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd5e908-2a61-457c-92ac-a6fd2f0fd38e None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:45.417121 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 63/122] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Oct 14 02:48:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 14 02:48:45.422296 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa012c29-8068-41b9-a3f7-ad3c5868be04 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:45.459806 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:45.487494 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:45.500754 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd5e908-2a61-457c-92ac-a6fd2f0fd38e None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:45.528063 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa012c29-8068-41b9-a3f7-ad3c5868be04 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:45.537795 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5073b7-9822-4983-96c4-5327ef79dff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:45.539092 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5073b7-9822-4983-96c4-5327ef79dff4 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:45.539453 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5073b7-9822-4983-96c4-5327ef79dff4 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:45.539804 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5073b7-9822-4983-96c4-5327ef79dff4 None admin] PATH_INFO: `/v3/users/2d06bb982e494934920981820968bc00` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:45.541022 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5073b7-9822-4983-96c4-5327ef79dff4 None admin] RBAC: Authorizing `identity:get_user(user_id=2d06bb982e494934920981820968bc00)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:45.542606 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5073b7-9822-4983-96c4-5327ef79dff4 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:45.544447 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 60/123] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Oct 14 02:48:45 2022] GET /identity/v3/users/2d06bb982e494934920981820968bc00 => generated 275 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:48:45.549503 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5073b7-9822-4983-96c4-5327ef79dff4 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:45.564682 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c28175-6b3d-4d2c-bfad-cb92c7539ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:45.565346 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c28175-6b3d-4d2c-bfad-cb92c7539ff5 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:45.565633 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c28175-6b3d-4d2c-bfad-cb92c7539ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:45.565983 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c28175-6b3d-4d2c-bfad-cb92c7539ff5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:45.566720 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c28175-6b3d-4d2c-bfad-cb92c7539ff5 None admin] RBAC: Adding query filter params (role.id=d695dc72739e4aa7a50d5fb73ad1a91f, scope.project.id=0b15574d2808402ca6940d47f10e2db8, user.id=2d06bb982e494934920981820968bc00) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:48:45.567031 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c28175-6b3d-4d2c-bfad-cb92c7539ff5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:45.568563 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c28175-6b3d-4d2c-bfad-cb92c7539ff5 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:45.575022 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 64/124] 10.222.0.39 () {56 vars in 1374 bytes} [Fri Oct 14 02:48:45 2022] GET /identity/v3/role_assignments?user.id=2d06bb982e494934920981820968bc00&scope.project.id=0b15574d2808402ca6940d47f10e2db8&role.id=d695dc72739e4aa7a50d5fb73ad1a91f => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:48:45.575705 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c28175-6b3d-4d2c-bfad-cb92c7539ff5 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:45.664407 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5073b7-9822-4983-96c4-5327ef79dff4 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:45.688621 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c28175-6b3d-4d2c-bfad-cb92c7539ff5 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:45.696011 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:45.701855 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac92ff1-c47a-40c3-aa99-20bf4ea02b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:45.702519 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac92ff1-c47a-40c3-aa99-20bf4ea02b64 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:45.702951 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac92ff1-c47a-40c3-aa99-20bf4ea02b64 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:45.703241 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac92ff1-c47a-40c3-aa99-20bf4ea02b64 None admin] PATH_INFO: `/v3/projects/0b15574d2808402ca6940d47f10e2db8` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:45.705057 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac92ff1-c47a-40c3-aa99-20bf4ea02b64 None admin] RBAC: Authorizing `identity:get_project(project_id=0b15574d2808402ca6940d47f10e2db8)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:45.709620 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac92ff1-c47a-40c3-aa99-20bf4ea02b64 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:45.709620 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 61/125] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Oct 14 02:48:45 2022] GET /identity/v3/projects/0b15574d2808402ca6940d47f10e2db8 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:48:45.718325 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:45.718822 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac92ff1-c47a-40c3-aa99-20bf4ea02b64 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:45.728136 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa586378-cfe9-4320-a229-0280b6114a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:45.728851 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa586378-cfe9-4320-a229-0280b6114a56 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:45.730066 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa586378-cfe9-4320-a229-0280b6114a56 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:45.730350 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa586378-cfe9-4320-a229-0280b6114a56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:45.865198 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac92ff1-c47a-40c3-aa99-20bf4ea02b64 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:45.873548 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa586378-cfe9-4320-a229-0280b6114a56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:45.875200 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa586378-cfe9-4320-a229-0280b6114a56 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:45.908156 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a2de0d-fb00-4b81-a72a-f99e15d4d7ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:45.908784 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a2de0d-fb00-4b81-a72a-f99e15d4d7ff None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:45.909071 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a2de0d-fb00-4b81-a72a-f99e15d4d7ff None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:45.909351 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a2de0d-fb00-4b81-a72a-f99e15d4d7ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:45.910243 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a2de0d-fb00-4b81-a72a-f99e15d4d7ff None admin] RBAC: Adding query filter params (role.id=9e86d182c0594e0785b72ba5f390a114, scope.project.id=0b15574d2808402ca6940d47f10e2db8, user.id=316b39adbf9f4b0d9c13dc4c11a24e73) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:48:45.910668 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a2de0d-fb00-4b81-a72a-f99e15d4d7ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:45.912126 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a2de0d-fb00-4b81-a72a-f99e15d4d7ff None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:45.922904 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 62/126] 10.222.0.39 () {56 vars in 1374 bytes} [Fri Oct 14 02:48:45 2022] GET /identity/v3/role_assignments?user.id=316b39adbf9f4b0d9c13dc4c11a24e73&scope.project.id=0b15574d2808402ca6940d47f10e2db8&role.id=9e86d182c0594e0785b72ba5f390a114 => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:48:45.997182 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 65/127] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Oct 14 02:48:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 14 02:48:46.003301 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a2de0d-fb00-4b81-a72a-f99e15d4d7ff None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:46.091268 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:46.133045 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a2de0d-fb00-4b81-a72a-f99e15d4d7ff None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:46.170018 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77976daa-90e7-451b-b43b-507c3254ed32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:46.170667 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77976daa-90e7-451b-b43b-507c3254ed32 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:46.170956 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77976daa-90e7-451b-b43b-507c3254ed32 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:46.171238 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77976daa-90e7-451b-b43b-507c3254ed32 None admin] PATH_INFO: `/v3/projects/ac970cbd2d634d1d9fa09ffd8070dc34` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:46.172374 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77976daa-90e7-451b-b43b-507c3254ed32 None admin] RBAC: Authorizing `identity:get_project(project_id=ac970cbd2d634d1d9fa09ffd8070dc34)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:46.173939 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77976daa-90e7-451b-b43b-507c3254ed32 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:46.175648 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 63/128] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Oct 14 02:48:46 2022] GET /identity/v3/projects/ac970cbd2d634d1d9fa09ffd8070dc34 => generated 312 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:48:46.181904 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa586378-cfe9-4320-a229-0280b6114a56 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:46.213443 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:46.291523 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa586378-cfe9-4320-a229-0280b6114a56 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:46.325186 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad8f29d-5dc4-4c24-b774-48a095e2c4ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:46.325816 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad8f29d-5dc4-4c24-b774-48a095e2c4ab None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:46.326103 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad8f29d-5dc4-4c24-b774-48a095e2c4ab None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:46.326377 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad8f29d-5dc4-4c24-b774-48a095e2c4ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:46.327096 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad8f29d-5dc4-4c24-b774-48a095e2c4ab None admin] RBAC: Adding query filter params (role.id=9bd0470b544240209a420f640c79c06c, scope.project.id=ac970cbd2d634d1d9fa09ffd8070dc34, user.id=2d06bb982e494934920981820968bc00) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:48:46.327402 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad8f29d-5dc4-4c24-b774-48a095e2c4ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:46.328842 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad8f29d-5dc4-4c24-b774-48a095e2c4ab None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:46.334538 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 66/129] 10.222.0.39 () {56 vars in 1374 bytes} [Fri Oct 14 02:48:46 2022] GET /identity/v3/role_assignments?user.id=2d06bb982e494934920981820968bc00&scope.project.id=ac970cbd2d634d1d9fa09ffd8070dc34&role.id=9bd0470b544240209a420f640c79c06c => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:48:46.965195 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc812f-ce29-43a8-bbee-194f0d5e2c85 None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:46.965489 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc812f-ce29-43a8-bbee-194f0d5e2c85 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:46.965799 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc812f-ce29-43a8-bbee-194f0d5e2c85 None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:46.967154 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 64/130] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:48:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:48:46.972681 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6495154f-77ab-41a7-85ce-218aabcf6ece None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:46.972972 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6495154f-77ab-41a7-85ce-218aabcf6ece None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:46.973250 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6495154f-77ab-41a7-85ce-218aabcf6ece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:46.991148 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-6495154f-77ab-41a7-85ce-218aabcf6ece None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:48:46.992821 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-6495154f-77ab-41a7-85ce-218aabcf6ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:48:47.058472 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 67/131] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:48:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:48:47.328574 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acc812f-ce29-43a8-bbee-194f0d5e2c85 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:47.329318 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c167708-e5b2-4eaf-aa85-78b4a88864fd None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:47.329611 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c167708-e5b2-4eaf-aa85-78b4a88864fd None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:47.329947 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c167708-e5b2-4eaf-aa85-78b4a88864fd None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:47.331319 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 68/132] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:48:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 14 02:48:47.336312 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad87542-bb23-4295-be4a-dfb146f2c516 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:47.336596 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad87542-bb23-4295-be4a-dfb146f2c516 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:47.336874 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad87542-bb23-4295-be4a-dfb146f2c516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:47.355487 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-bad87542-bb23-4295-be4a-dfb146f2c516 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:48:47.356659 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-bad87542-bb23-4295-be4a-dfb146f2c516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:48:47.370926 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:47.422444 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 69/133] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:48:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 14 02:48:47.430471 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acc812f-ce29-43a8-bbee-194f0d5e2c85 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:47.457931 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:47.462614 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23486b78-ce07-4217-802e-91a1e4e86353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:47.463248 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23486b78-ce07-4217-802e-91a1e4e86353 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:47.463534 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23486b78-ce07-4217-802e-91a1e4e86353 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:47.463814 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23486b78-ce07-4217-802e-91a1e4e86353 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:47.464918 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23486b78-ce07-4217-802e-91a1e4e86353 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:47.466420 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23486b78-ce07-4217-802e-91a1e4e86353 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:47.467989 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 65/134] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:48:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:48:47.473339 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad87542-bb23-4295-be4a-dfb146f2c516 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:47.569090 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad87542-bb23-4295-be4a-dfb146f2c516 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:47.601464 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd49d0f-f2ba-4984-8f83-56b5558eeb14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:47.602034 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd49d0f-f2ba-4984-8f83-56b5558eeb14 None admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:47.602315 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd49d0f-f2ba-4984-8f83-56b5558eeb14 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:47.602584 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd49d0f-f2ba-4984-8f83-56b5558eeb14 None admin] PATH_INFO: `/v3/users` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:47.603225 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd49d0f-f2ba-4984-8f83-56b5558eeb14 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:47.604926 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd49d0f-f2ba-4984-8f83-56b5558eeb14 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:47.675702 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 70/135] 10.222.0.39 () {60 vars in 1132 bytes} [Fri Oct 14 02:48:47 2022] POST /identity/v3/users => generated 283 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:48:47.688384 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23486b78-ce07-4217-802e-91a1e4e86353 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:47.792997 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23486b78-ce07-4217-802e-91a1e4e86353 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:47.827754 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e207902-fb26-4ca0-937c-f32a071d12dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:47.828409 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e207902-fb26-4ca0-937c-f32a071d12dc None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:47.828698 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e207902-fb26-4ca0-937c-f32a071d12dc None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:47.828979 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e207902-fb26-4ca0-937c-f32a071d12dc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:47.840327 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e207902-fb26-4ca0-937c-f32a071d12dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:48:47.845740 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-9e207902-fb26-4ca0-937c-f32a071d12dc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 14 02:48:47.845740 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:48:47.845740 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:48:47.845740 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:48:47.845740 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:48:47.845740 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:48:47.845740 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:48:47.845740 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:48:47.845740 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:48:47.845740 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:48:47.845740 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:48:47.845740 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:48:47.845740 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:48:47.845740 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:48:47.845740 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:48:47.845740 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:48:47.846601 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:48:47.846601 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:48:47.846601 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:48:47.846601 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:48:47.846601 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:48:47.846601 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:48:47.846601 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:48:47.846601 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:48:47.846601 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:48:47.846601 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:48:47.846601 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:48:47.846601 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:48:47.846601 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:48:47.846601 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:48:47.846601 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:48:47.846601 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:48:47.847356 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:48:47.847356 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:48:47.847356 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:48:47.847356 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:48:47.847356 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:48:47.847356 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:48:47.847356 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:48:47.847356 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:48:47.847356 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:48:47.847356 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:48:47.847356 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 14 02:48:47.847356 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:48:47.847356 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 66/136] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Oct 14 02:48:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:48:47.853393 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd49d0f-f2ba-4984-8f83-56b5558eeb14 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:47.863162 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:47.958587 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd49d0f-f2ba-4984-8f83-56b5558eeb14 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:47.997583 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d53ee3-8fb6-42ad-a5a4-db44d58070c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:47.998267 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d53ee3-8fb6-42ad-a5a4-db44d58070c3 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:47.998549 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d53ee3-8fb6-42ad-a5a4-db44d58070c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:47.998822 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d53ee3-8fb6-42ad-a5a4-db44d58070c3 None admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:47.999487 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d53ee3-8fb6-42ad-a5a4-db44d58070c3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:48:47.999794 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d53ee3-8fb6-42ad-a5a4-db44d58070c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:48.001208 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d53ee3-8fb6-42ad-a5a4-db44d58070c3 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:48.011867 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 71/137] 10.222.0.39 () {56 vars in 1101 bytes} [Fri Oct 14 02:48:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:48:48.017469 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e207902-fb26-4ca0-937c-f32a071d12dc None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:48.082897 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:48.128987 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e207902-fb26-4ca0-937c-f32a071d12dc None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:48.168385 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55fee3e-30ec-44d5-b3f9-ce13f41776b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:48.169026 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55fee3e-30ec-44d5-b3f9-ce13f41776b8 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:48.169308 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55fee3e-30ec-44d5-b3f9-ce13f41776b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:48.169584 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55fee3e-30ec-44d5-b3f9-ce13f41776b8 None admin] PATH_INFO: `/v3/users/e4d39d44cb9e4196ada8e0e4e4e5a43f` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:48.170748 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55fee3e-30ec-44d5-b3f9-ce13f41776b8 None admin] RBAC: Authorizing `identity:get_user(user_id=e4d39d44cb9e4196ada8e0e4e4e5a43f)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:48.172212 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55fee3e-30ec-44d5-b3f9-ce13f41776b8 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:48.173820 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 67/138] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Oct 14 02:48:48 2022] GET /identity/v3/users/e4d39d44cb9e4196ada8e0e4e4e5a43f => generated 289 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:48:48.179414 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d53ee3-8fb6-42ad-a5a4-db44d58070c3 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:48.249235 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:48.285380 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d53ee3-8fb6-42ad-a5a4-db44d58070c3 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:48.320786 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a530ebfb-3511-4cb8-a041-5cc474754ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:48.321740 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a530ebfb-3511-4cb8-a041-5cc474754ee4 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:48.322354 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a530ebfb-3511-4cb8-a041-5cc474754ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:48.322850 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a530ebfb-3511-4cb8-a041-5cc474754ee4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:48.393053 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a530ebfb-3511-4cb8-a041-5cc474754ee4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:48.394870 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a530ebfb-3511-4cb8-a041-5cc474754ee4 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:48.465017 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:48.500164 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 72/139] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Oct 14 02:48:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 14 02:48:48.505788 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55fee3e-30ec-44d5-b3f9-ce13f41776b8 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:48.609195 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55fee3e-30ec-44d5-b3f9-ce13f41776b8 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:48.635395 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:48.643019 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44893bba-b2ad-4239-88ad-b392fbc879a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:48.643761 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44893bba-b2ad-4239-88ad-b392fbc879a6 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:48.644317 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44893bba-b2ad-4239-88ad-b392fbc879a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:48.644649 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44893bba-b2ad-4239-88ad-b392fbc879a6 None admin] PATH_INFO: `/v3/projects/0b15574d2808402ca6940d47f10e2db8` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:48.645853 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44893bba-b2ad-4239-88ad-b392fbc879a6 None admin] RBAC: Authorizing `identity:get_project(project_id=0b15574d2808402ca6940d47f10e2db8)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:48.647378 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44893bba-b2ad-4239-88ad-b392fbc879a6 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:48.649985 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 68/140] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Oct 14 02:48:48 2022] GET /identity/v3/projects/0b15574d2808402ca6940d47f10e2db8 => generated 298 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:48:48.655793 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a530ebfb-3511-4cb8-a041-5cc474754ee4 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:48.759174 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a530ebfb-3511-4cb8-a041-5cc474754ee4 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:48.796024 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21649a67-b0be-41ba-bf4e-48fb75ef5305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:48.796685 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21649a67-b0be-41ba-bf4e-48fb75ef5305 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:48.797030 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21649a67-b0be-41ba-bf4e-48fb75ef5305 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:48.797361 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21649a67-b0be-41ba-bf4e-48fb75ef5305 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:48.798120 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21649a67-b0be-41ba-bf4e-48fb75ef5305 None admin] RBAC: Adding query filter params (role.id=3026af19537b46cab1318d0e13ae4163, scope.project.id=0b15574d2808402ca6940d47f10e2db8, user.id=e4d39d44cb9e4196ada8e0e4e4e5a43f) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:48:48.798480 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21649a67-b0be-41ba-bf4e-48fb75ef5305 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:48.799944 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21649a67-b0be-41ba-bf4e-48fb75ef5305 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:48.806369 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 73/141] 10.222.0.39 () {56 vars in 1374 bytes} [Fri Oct 14 02:48:48 2022] GET /identity/v3/role_assignments?user.id=e4d39d44cb9e4196ada8e0e4e4e5a43f&scope.project.id=0b15574d2808402ca6940d47f10e2db8&role.id=3026af19537b46cab1318d0e13ae4163 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:48:49.936033 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f68b995-f8fb-407a-94fb-7a50f40267b2 None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:49.936539 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f68b995-f8fb-407a-94fb-7a50f40267b2 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:49.936612 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f68b995-f8fb-407a-94fb-7a50f40267b2 None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:49.938147 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 69/142] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:48:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:48:49.943392 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c25005-bbba-4713-8816-d30bc31eb1a9 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:49.943783 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c25005-bbba-4713-8816-d30bc31eb1a9 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:49.943947 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c25005-bbba-4713-8816-d30bc31eb1a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:49.961708 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-f4c25005-bbba-4713-8816-d30bc31eb1a9 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:48:49.962914 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-f4c25005-bbba-4713-8816-d30bc31eb1a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:48:49.989619 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:50.034869 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 74/143] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:48:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:48:50.118115 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126387e7-9e30-4f08-890e-e01a1f864383 None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:50.118397 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126387e7-9e30-4f08-890e-e01a1f864383 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:50.118680 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126387e7-9e30-4f08-890e-e01a1f864383 None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:50.120231 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 70/144] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:48:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:48:50.125090 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb4659e-3478-4594-95d5-26f12bcc66ce None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:50.125382 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb4659e-3478-4594-95d5-26f12bcc66ce None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:50.125662 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb4659e-3478-4594-95d5-26f12bcc66ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:50.144167 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-cfb4659e-3478-4594-95d5-26f12bcc66ce None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:48:50.145392 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-cfb4659e-3478-4594-95d5-26f12bcc66ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:48:50.216894 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 75/145] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:48:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:48:50.266603 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c0de31-cd99-4677-b5c1-bbb6c1732f0f None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:50.266933 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c0de31-cd99-4677-b5c1-bbb6c1732f0f None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:50.267162 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c0de31-cd99-4677-b5c1-bbb6c1732f0f None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:50.268623 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 71/146] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:48:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:48:50.274129 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3815fe3-b5d9-4ea2-9153-8a7fb26111c9 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:50.274548 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3815fe3-b5d9-4ea2-9153-8a7fb26111c9 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:50.274872 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3815fe3-b5d9-4ea2-9153-8a7fb26111c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:50.293398 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-a3815fe3-b5d9-4ea2-9153-8a7fb26111c9 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:48:50.294598 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-a3815fe3-b5d9-4ea2-9153-8a7fb26111c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:48:50.304650 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c0de31-cd99-4677-b5c1-bbb6c1732f0f None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:50.364217 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 76/147] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:48:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 14 02:48:50.412134 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c0de31-cd99-4677-b5c1-bbb6c1732f0f None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:50.445609 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84b9c49-6f77-49e6-9dfa-1653f593e9cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:50.446271 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84b9c49-6f77-49e6-9dfa-1653f593e9cc None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:50.446554 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84b9c49-6f77-49e6-9dfa-1653f593e9cc None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:50.446829 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84b9c49-6f77-49e6-9dfa-1653f593e9cc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:50.457316 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84b9c49-6f77-49e6-9dfa-1653f593e9cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:48:50.463269 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-c84b9c49-6f77-49e6-9dfa-1653f593e9cc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:48:50.463269 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:48:50.463269 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:48:50.463269 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:48:50.463269 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:48:50.463269 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:48:50.463269 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:48:50.463269 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:48:50.463269 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:48:50.463269 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:48:50.463269 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:48:50.463269 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:48:50.463269 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:48:50.463269 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:48:50.463269 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:48:50.463269 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:48:50.464316 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:48:50.464316 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:48:50.464316 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:48:50.464316 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:48:50.464316 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:48:50.464316 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:48:50.464316 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:48:50.464316 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:48:50.464316 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:48:50.464316 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:48:50.464316 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:48:50.464316 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:48:50.464316 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:48:50.464316 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:48:50.464316 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:48:50.464316 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:48:50.466850 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:48:50.466850 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:48:50.466850 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:48:50.466850 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:48:50.466850 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:48:50.466850 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:48:50.466850 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:48:50.466850 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:48:50.466850 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:48:50.466850 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:48:50.466850 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:48:50.466850 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:48:50.466850 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 72/148] 10.222.0.39 () {56 vars in 1096 bytes} [Fri Oct 14 02:48:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 02:48:50.470589 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3815fe3-b5d9-4ea2-9153-8a7fb26111c9 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:50.491292 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84b9c49-6f77-49e6-9dfa-1653f593e9cc None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:50.495883 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:50.553743 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:50.572682 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3815fe3-b5d9-4ea2-9153-8a7fb26111c9 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:50.602986 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84b9c49-6f77-49e6-9dfa-1653f593e9cc None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:50.607584 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12735a4-f431-4a1e-84a3-026593e34e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:50.608194 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12735a4-f431-4a1e-84a3-026593e34e29 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:50.608477 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12735a4-f431-4a1e-84a3-026593e34e29 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:50.608753 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12735a4-f431-4a1e-84a3-026593e34e29 None admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:50.609419 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12735a4-f431-4a1e-84a3-026593e34e29 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:48:50.609725 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12735a4-f431-4a1e-84a3-026593e34e29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:50.611145 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12735a4-f431-4a1e-84a3-026593e34e29 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:50.619538 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 77/149] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Oct 14 02:48:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:48:50.624947 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12735a4-f431-4a1e-84a3-026593e34e29 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:50.638683 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb44f9b-cd2c-460e-904f-54efbbfe24cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:50.639319 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb44f9b-cd2c-460e-904f-54efbbfe24cd None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:50.639602 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb44f9b-cd2c-460e-904f-54efbbfe24cd None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:50.639879 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb44f9b-cd2c-460e-904f-54efbbfe24cd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:50.651248 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb44f9b-cd2c-460e-904f-54efbbfe24cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:48:50.656517 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-eeb44f9b-cd2c-460e-904f-54efbbfe24cd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 14 02:48:50.656517 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:48:50.656517 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:48:50.656517 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:48:50.656517 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:48:50.656517 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:48:50.656517 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:48:50.656517 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:48:50.656517 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:48:50.656517 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:48:50.656517 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:48:50.656517 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:48:50.656517 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:48:50.656517 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:48:50.656517 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:48:50.656517 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:48:50.657639 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:48:50.657639 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:48:50.657639 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:48:50.657639 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:48:50.657639 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:48:50.657639 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:48:50.657639 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:48:50.657639 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:48:50.657639 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:48:50.657639 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:48:50.657639 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:48:50.657639 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:48:50.657639 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:48:50.657639 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:48:50.657639 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:48:50.657639 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:48:50.658649 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:48:50.658649 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:48:50.658649 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:48:50.658649 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:48:50.658649 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:48:50.658649 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:48:50.658649 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:48:50.658649 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:48:50.658649 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:48:50.658649 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:48:50.658649 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 14 02:48:50.658649 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:48:50.658649 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 73/150] 10.222.0.39 () {56 vars in 1114 bytes} [Fri Oct 14 02:48:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 02:48:50.658649 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb44f9b-cd2c-460e-904f-54efbbfe24cd None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:50.710991 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:50.726018 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12735a4-f431-4a1e-84a3-026593e34e29 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:50.757707 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb44f9b-cd2c-460e-904f-54efbbfe24cd None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:50.760337 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca86e390-677c-4786-8f7a-382a8bb5110d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:50.760961 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca86e390-677c-4786-8f7a-382a8bb5110d None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:50.761246 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca86e390-677c-4786-8f7a-382a8bb5110d None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:50.761526 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca86e390-677c-4786-8f7a-382a8bb5110d None admin] PATH_INFO: `/v3/users/09320cb2423c4c1689a3fbca9ff22266` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:50.773001 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:50.775124 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca86e390-677c-4786-8f7a-382a8bb5110d None admin] RBAC: Authorizing `identity:get_user(user_id=09320cb2423c4c1689a3fbca9ff22266)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:50.776588 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca86e390-677c-4786-8f7a-382a8bb5110d None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:50.778299 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 78/151] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Oct 14 02:48:50 2022] GET /identity/v3/users/09320cb2423c4c1689a3fbca9ff22266 => generated 283 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:48:50.783102 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca86e390-677c-4786-8f7a-382a8bb5110d None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:50.791464 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d89f4b-4e1b-48b5-8407-78a24b2dd67c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:50.792097 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d89f4b-4e1b-48b5-8407-78a24b2dd67c None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:50.792378 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d89f4b-4e1b-48b5-8407-78a24b2dd67c None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:50.792823 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d89f4b-4e1b-48b5-8407-78a24b2dd67c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:50.794361 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d89f4b-4e1b-48b5-8407-78a24b2dd67c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:50.796065 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d89f4b-4e1b-48b5-8407-78a24b2dd67c None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:50.797639 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 74/152] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:48:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:48:50.798453 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d89f4b-4e1b-48b5-8407-78a24b2dd67c None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:50.878925 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca86e390-677c-4786-8f7a-382a8bb5110d None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:50.897199 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d89f4b-4e1b-48b5-8407-78a24b2dd67c None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:50.912394 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d742acb-3f59-4a91-84a4-ca58f72f1fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:50.913024 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d742acb-3f59-4a91-84a4-ca58f72f1fa3 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:50.913342 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d742acb-3f59-4a91-84a4-ca58f72f1fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:50.913618 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d742acb-3f59-4a91-84a4-ca58f72f1fa3 None admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:50.914310 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d742acb-3f59-4a91-84a4-ca58f72f1fa3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:48:50.914616 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d742acb-3f59-4a91-84a4-ca58f72f1fa3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:50.915990 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d742acb-3f59-4a91-84a4-ca58f72f1fa3 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:50.923652 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 79/153] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Oct 14 02:48:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:48:50.924409 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d742acb-3f59-4a91-84a4-ca58f72f1fa3 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:50.929892 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:50.930807 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e6f9b7-3bd6-4342-b78f-6a728e4ab54b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:50.931374 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e6f9b7-3bd6-4342-b78f-6a728e4ab54b None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:50.931654 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e6f9b7-3bd6-4342-b78f-6a728e4ab54b None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:50.931931 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e6f9b7-3bd6-4342-b78f-6a728e4ab54b None admin] PATH_INFO: `/v3/projects/b5c28654bb91452aaa2aff4568067821` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:50.933055 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e6f9b7-3bd6-4342-b78f-6a728e4ab54b None admin] RBAC: Authorizing `identity:get_project(project_id=b5c28654bb91452aaa2aff4568067821)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:50.934539 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e6f9b7-3bd6-4342-b78f-6a728e4ab54b None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:50.936150 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 75/154] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Oct 14 02:48:50 2022] GET /identity/v3/projects/b5c28654bb91452aaa2aff4568067821 => generated 302 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:48:50.936927 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e6f9b7-3bd6-4342-b78f-6a728e4ab54b None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:50.990193 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:51.028277 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d742acb-3f59-4a91-84a4-ca58f72f1fa3 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:51.037980 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e6f9b7-3bd6-4342-b78f-6a728e4ab54b None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:51.061508 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a284c3f-09de-4b57-85ce-ab4e741f8f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:51.062139 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a284c3f-09de-4b57-85ce-ab4e741f8f28 None admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:51.062423 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a284c3f-09de-4b57-85ce-ab4e741f8f28 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:51.062700 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a284c3f-09de-4b57-85ce-ab4e741f8f28 None admin] PATH_INFO: `/v3/users` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:51.063343 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a284c3f-09de-4b57-85ce-ab4e741f8f28 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:51.064787 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a284c3f-09de-4b57-85ce-ab4e741f8f28 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:51.071189 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a49f05-baea-45f8-8f48-20a180fd5c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:51.071811 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a49f05-baea-45f8-8f48-20a180fd5c96 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:51.072088 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a49f05-baea-45f8-8f48-20a180fd5c96 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:51.072355 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a49f05-baea-45f8-8f48-20a180fd5c96 None admin] PATH_INFO: `/v3/users/09320cb2423c4c1689a3fbca9ff22266` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:51.073464 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a49f05-baea-45f8-8f48-20a180fd5c96 None admin] RBAC: Authorizing `identity:get_user(user_id=09320cb2423c4c1689a3fbca9ff22266)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:51.074921 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a49f05-baea-45f8-8f48-20a180fd5c96 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:51.076645 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 76/155] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Oct 14 02:48:50 2022] GET /identity/v3/users/09320cb2423c4c1689a3fbca9ff22266 => generated 283 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:48:51.077290 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a49f05-baea-45f8-8f48-20a180fd5c96 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:51.173085 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a49f05-baea-45f8-8f48-20a180fd5c96 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:51.185570 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 80/156] 10.222.0.39 () {60 vars in 1132 bytes} [Fri Oct 14 02:48:50 2022] POST /identity/v3/users => generated 297 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:48:51.186453 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a284c3f-09de-4b57-85ce-ab4e741f8f28 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:51.195738 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:51.212213 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48793fd4-c26b-41f3-9a11-236c47a16e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:51.212938 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48793fd4-c26b-41f3-9a11-236c47a16e48 None admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:51.213221 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48793fd4-c26b-41f3-9a11-236c47a16e48 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:51.213499 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48793fd4-c26b-41f3-9a11-236c47a16e48 None admin] PATH_INFO: `/v3/projects/b5c28654bb91452aaa2aff4568067821/users/09320cb2423c4c1689a3fbca9ff22266/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:51.215606 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48793fd4-c26b-41f3-9a11-236c47a16e48 None admin] RBAC: Authorizing `identity:create_grant(project_id=b5c28654bb91452aaa2aff4568067821, user_id=09320cb2423c4c1689a3fbca9ff22266, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:51.217081 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48793fd4-c26b-41f3-9a11-236c47a16e48 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:51.255419 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:51.289294 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 77/157] 10.222.0.39 () {58 vars in 1439 bytes} [Fri Oct 14 02:48:51 2022] PUT /identity/v3/projects/b5c28654bb91452aaa2aff4568067821/users/09320cb2423c4c1689a3fbca9ff22266/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:48:51.306862 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a284c3f-09de-4b57-85ce-ab4e741f8f28 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:51.346850 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1df5a9-0905-4ebc-87b4-d91f6caa58f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:51.347526 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1df5a9-0905-4ebc-87b4-d91f6caa58f7 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:51.347812 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1df5a9-0905-4ebc-87b4-d91f6caa58f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:51.348089 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1df5a9-0905-4ebc-87b4-d91f6caa58f7 None admin] PATH_INFO: `/v3/projects/b5c28654bb91452aaa2aff4568067821` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:51.349442 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1df5a9-0905-4ebc-87b4-d91f6caa58f7 None admin] RBAC: Authorizing `identity:get_project(project_id=b5c28654bb91452aaa2aff4568067821)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:51.351535 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1df5a9-0905-4ebc-87b4-d91f6caa58f7 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:51.353528 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 81/158] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Oct 14 02:48:51 2022] GET /identity/v3/projects/b5c28654bb91452aaa2aff4568067821 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:48:51.359643 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48793fd4-c26b-41f3-9a11-236c47a16e48 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:51.474587 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48793fd4-c26b-41f3-9a11-236c47a16e48 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:51.513789 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38d8260-3231-4ef3-884c-ef0ac3dda9bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:51.514395 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d8260-3231-4ef3-884c-ef0ac3dda9bd None admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:51.514679 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d8260-3231-4ef3-884c-ef0ac3dda9bd None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:51.514957 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d8260-3231-4ef3-884c-ef0ac3dda9bd None admin] PATH_INFO: `/v3/projects/b5c28654bb91452aaa2aff4568067821/users/09320cb2423c4c1689a3fbca9ff22266/roles/d695dc72739e4aa7a50d5fb73ad1a91f` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:51.517102 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38d8260-3231-4ef3-884c-ef0ac3dda9bd None admin] RBAC: Authorizing `identity:create_grant(project_id=b5c28654bb91452aaa2aff4568067821, user_id=09320cb2423c4c1689a3fbca9ff22266, role_id=d695dc72739e4aa7a50d5fb73ad1a91f)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:51.518632 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38d8260-3231-4ef3-884c-ef0ac3dda9bd None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:51.614824 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 78/159] 10.222.0.39 () {58 vars in 1439 bytes} [Fri Oct 14 02:48:51 2022] PUT /identity/v3/projects/b5c28654bb91452aaa2aff4568067821/users/09320cb2423c4c1689a3fbca9ff22266/roles/d695dc72739e4aa7a50d5fb73ad1a91f => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:48:53.478365 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6049a6e0-39a0-458a-b1f5-7ac5ddf699cf None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:53.478975 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6049a6e0-39a0-458a-b1f5-7ac5ddf699cf None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:53.478975 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6049a6e0-39a0-458a-b1f5-7ac5ddf699cf None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:53.480749 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 82/160] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:48:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:48:53.486264 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc5ea40-6c3b-4783-88de-874ee6aae1ea None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:53.487083 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc5ea40-6c3b-4783-88de-874ee6aae1ea None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:53.487587 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc5ea40-6c3b-4783-88de-874ee6aae1ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:53.506256 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-4bc5ea40-6c3b-4783-88de-874ee6aae1ea None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:48:53.508321 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-4bc5ea40-6c3b-4783-88de-874ee6aae1ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:48:53.535959 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5275d5-ac40-43f8-a7be-47f224d22e42 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:53.536574 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5275d5-ac40-43f8-a7be-47f224d22e42 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:53.537062 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5275d5-ac40-43f8-a7be-47f224d22e42 None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:53.539146 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 83/161] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:48:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 14 02:48:53.548801 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254362ed-f105-416b-8d7e-057ce64f7dc4 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:53.549083 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254362ed-f105-416b-8d7e-057ce64f7dc4 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:53.549354 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254362ed-f105-416b-8d7e-057ce64f7dc4 None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:53.550673 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 84/162] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:48:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 14 02:48:53.551715 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:53.553191 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc76c41-6bdb-4f23-892e-f2690b6f071d None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:53.554274 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc76c41-6bdb-4f23-892e-f2690b6f071d None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:53.554773 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc76c41-6bdb-4f23-892e-f2690b6f071d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:53.573434 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-7fc76c41-6bdb-4f23-892e-f2690b6f071d None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:48:53.574648 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-7fc76c41-6bdb-4f23-892e-f2690b6f071d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:48:53.583941 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 79/163] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:48:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:48:53.587079 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda4503d-1bab-4016-8e86-b2e4ffa88587 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:53.587366 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda4503d-1bab-4016-8e86-b2e4ffa88587 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:53.587639 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda4503d-1bab-4016-8e86-b2e4ffa88587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:53.606801 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-cda4503d-1bab-4016-8e86-b2e4ffa88587 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:48:53.608153 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-cda4503d-1bab-4016-8e86-b2e4ffa88587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:48:53.658245 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 85/164] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:48:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:48:53.691174 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 80/165] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:48:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:48:53.773346 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51a6c4c-c6a0-465c-879e-5ca2d93ba607 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:53.773625 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51a6c4c-c6a0-465c-879e-5ca2d93ba607 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:53.773850 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51a6c4c-c6a0-465c-879e-5ca2d93ba607 None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:53.775360 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 86/166] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:48:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:48:53.783008 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe477ad-9bb3-45c5-b2ec-96ec7b9994dc None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:53.783541 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe477ad-9bb3-45c5-b2ec-96ec7b9994dc None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:53.784087 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe477ad-9bb3-45c5-b2ec-96ec7b9994dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:53.803338 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-afe477ad-9bb3-45c5-b2ec-96ec7b9994dc None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:48:53.804871 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-afe477ad-9bb3-45c5-b2ec-96ec7b9994dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:48:53.860870 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51a6c4c-c6a0-465c-879e-5ca2d93ba607 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:53.875244 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 81/167] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:48:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:48:53.922681 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:53.937976 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe477ad-9bb3-45c5-b2ec-96ec7b9994dc None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:53.967083 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51a6c4c-c6a0-465c-879e-5ca2d93ba607 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:53.970623 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:54.005551 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1d4f9a-99a7-4084-9e9a-975e49a0a612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:54.006460 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1d4f9a-99a7-4084-9e9a-975e49a0a612 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:54.006808 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1d4f9a-99a7-4084-9e9a-975e49a0a612 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:54.009084 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1d4f9a-99a7-4084-9e9a-975e49a0a612 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:54.021951 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1d4f9a-99a7-4084-9e9a-975e49a0a612 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:48:54.028997 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-7f1d4f9a-99a7-4084-9e9a-975e49a0a612 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:48:54.028997 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:48:54.028997 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:48:54.028997 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:48:54.028997 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:48:54.028997 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:48:54.028997 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:48:54.028997 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:48:54.028997 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:48:54.028997 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:48:54.028997 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:48:54.028997 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:48:54.028997 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:48:54.028997 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:48:54.028997 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:48:54.028997 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:48:54.029907 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:48:54.029907 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:48:54.029907 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:48:54.029907 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:48:54.029907 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:48:54.029907 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:48:54.029907 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:48:54.029907 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:48:54.029907 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:48:54.029907 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:48:54.029907 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:48:54.029907 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:48:54.029907 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:48:54.029907 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:48:54.029907 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:48:54.029907 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:48:54.030731 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:48:54.030731 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:48:54.030731 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:48:54.030731 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:48:54.030731 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:48:54.030731 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:48:54.030731 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:48:54.030731 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:48:54.030731 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:48:54.030731 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:48:54.030731 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:48:54.030731 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:48:54.030731 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 87/168] 10.222.0.39 () {56 vars in 1096 bytes} [Fri Oct 14 02:48:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 02:48:54.031496 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1d4f9a-99a7-4084-9e9a-975e49a0a612 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:54.050866 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe477ad-9bb3-45c5-b2ec-96ec7b9994dc None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:54.085369 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea692192-43f0-4b8e-b782-5c203751ceca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:54.086040 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea692192-43f0-4b8e-b782-5c203751ceca None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:54.086319 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea692192-43f0-4b8e-b782-5c203751ceca None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:54.086588 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea692192-43f0-4b8e-b782-5c203751ceca None admin] PATH_INFO: `/v3/roles/member` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:54.096348 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea692192-43f0-4b8e-b782-5c203751ceca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:48:54.101629 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-ea692192-43f0-4b8e-b782-5c203751ceca None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 14 02:48:54.101629 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:48:54.101629 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:48:54.101629 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:48:54.101629 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:48:54.101629 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:48:54.101629 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:48:54.101629 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:48:54.101629 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:48:54.101629 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:48:54.101629 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:48:54.101629 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:48:54.101629 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:48:54.101629 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:48:54.101629 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:48:54.101629 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:48:54.102518 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:48:54.102518 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:48:54.102518 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:48:54.102518 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:48:54.102518 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:48:54.102518 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:48:54.102518 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:48:54.102518 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:48:54.102518 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:48:54.102518 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:48:54.102518 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:48:54.102518 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:48:54.102518 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:48:54.102518 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:48:54.102518 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:48:54.102518 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:48:54.103288 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:48:54.103288 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:48:54.103288 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:48:54.103288 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:48:54.103288 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:48:54.103288 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:48:54.103288 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:48:54.103288 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:48:54.103288 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:48:54.103288 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:48:54.103288 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 14 02:48:54.103288 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:48:54.103288 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 82/169] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Oct 14 02:48:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 02:48:54.103969 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea692192-43f0-4b8e-b782-5c203751ceca None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:54.139455 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1d4f9a-99a7-4084-9e9a-975e49a0a612 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:54.157173 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:54.173682 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8b8c59-a121-481e-99ff-966bc5bca488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:54.174343 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b8c59-a121-481e-99ff-966bc5bca488 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:54.174629 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b8c59-a121-481e-99ff-966bc5bca488 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:54.174914 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b8c59-a121-481e-99ff-966bc5bca488 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:54.176013 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8b8c59-a121-481e-99ff-966bc5bca488 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:54.177518 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8b8c59-a121-481e-99ff-966bc5bca488 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:54.179175 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 88/170] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:48:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:48:54.179942 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8b8c59-a121-481e-99ff-966bc5bca488 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:54.238485 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:54.248221 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea692192-43f0-4b8e-b782-5c203751ceca None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:54.281833 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8b8c59-a121-481e-99ff-966bc5bca488 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:54.284160 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349f43d4-0141-4e06-a203-f3be094eb902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:54.284763 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349f43d4-0141-4e06-a203-f3be094eb902 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:54.285041 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349f43d4-0141-4e06-a203-f3be094eb902 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:54.285308 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349f43d4-0141-4e06-a203-f3be094eb902 None admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:54.285991 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349f43d4-0141-4e06-a203-f3be094eb902 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:48:54.286295 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349f43d4-0141-4e06-a203-f3be094eb902 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:54.290931 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349f43d4-0141-4e06-a203-f3be094eb902 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:54.295610 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 83/171] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Oct 14 02:48:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:48:54.296397 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349f43d4-0141-4e06-a203-f3be094eb902 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:54.317473 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f59914-3614-4b47-ba8b-6798b8a9529f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:54.318113 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f59914-3614-4b47-ba8b-6798b8a9529f None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:54.318476 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f59914-3614-4b47-ba8b-6798b8a9529f None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:54.322081 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f59914-3614-4b47-ba8b-6798b8a9529f None admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:54.322081 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f59914-3614-4b47-ba8b-6798b8a9529f None admin] RBAC: Adding query filter params (name=member) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:48:54.322081 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f59914-3614-4b47-ba8b-6798b8a9529f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:54.322081 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f59914-3614-4b47-ba8b-6798b8a9529f None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:54.329796 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 89/172] 10.222.0.39 () {56 vars in 1101 bytes} [Fri Oct 14 02:48:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:48:54.330415 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f59914-3614-4b47-ba8b-6798b8a9529f None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:54.378797 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:54.400208 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349f43d4-0141-4e06-a203-f3be094eb902 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:54.429571 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f59914-3614-4b47-ba8b-6798b8a9529f None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:54.432473 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037b6894-5aed-4fa1-b0e3-e7225a522882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:54.433375 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037b6894-5aed-4fa1-b0e3-e7225a522882 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:54.433669 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037b6894-5aed-4fa1-b0e3-e7225a522882 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:54.433971 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037b6894-5aed-4fa1-b0e3-e7225a522882 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:54.443993 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037b6894-5aed-4fa1-b0e3-e7225a522882 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:48:54.449845 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-037b6894-5aed-4fa1-b0e3-e7225a522882 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 14 02:48:54.449845 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:48:54.449845 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:48:54.449845 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:48:54.449845 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:48:54.449845 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:48:54.449845 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:48:54.449845 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:48:54.449845 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:48:54.449845 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:48:54.449845 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:48:54.449845 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:48:54.449845 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:48:54.449845 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:48:54.449845 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:48:54.449845 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:48:54.450711 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:48:54.450711 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:48:54.450711 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:48:54.450711 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:48:54.450711 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:48:54.450711 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:48:54.450711 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:48:54.450711 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:48:54.450711 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:48:54.450711 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:48:54.450711 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:48:54.450711 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:48:54.450711 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:48:54.450711 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:48:54.450711 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:48:54.450711 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:48:54.451716 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:48:54.451716 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:48:54.451716 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:48:54.451716 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:48:54.451716 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:48:54.451716 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:48:54.451716 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:48:54.451716 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:48:54.451716 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:48:54.451716 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:48:54.451716 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 14 02:48:54.451716 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:48:54.451716 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 84/173] 10.222.0.39 () {56 vars in 1114 bytes} [Fri Oct 14 02:48:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:48:54.456490 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037b6894-5aed-4fa1-b0e3-e7225a522882 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:54.468213 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712f2e0a-aeda-4779-bd43-b96a8652e211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:54.468818 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712f2e0a-aeda-4779-bd43-b96a8652e211 None admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:54.469105 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712f2e0a-aeda-4779-bd43-b96a8652e211 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:54.469387 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712f2e0a-aeda-4779-bd43-b96a8652e211 None admin] PATH_INFO: `/v3/users` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:54.470061 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712f2e0a-aeda-4779-bd43-b96a8652e211 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:54.471521 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712f2e0a-aeda-4779-bd43-b96a8652e211 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:54.471713 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:54.552852 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037b6894-5aed-4fa1-b0e3-e7225a522882 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:54.561792 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 90/174] 10.222.0.39 () {60 vars in 1132 bytes} [Fri Oct 14 02:48:54 2022] POST /identity/v3/users => generated 297 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:48:54.562556 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712f2e0a-aeda-4779-bd43-b96a8652e211 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:54.591949 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9747aa-2c7b-4091-b22a-2ad18a761799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:54.592597 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9747aa-2c7b-4091-b22a-2ad18a761799 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:54.592879 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9747aa-2c7b-4091-b22a-2ad18a761799 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:54.593154 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9747aa-2c7b-4091-b22a-2ad18a761799 None admin] PATH_INFO: `/v3/users/09320cb2423c4c1689a3fbca9ff22266` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:54.594402 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9747aa-2c7b-4091-b22a-2ad18a761799 None admin] RBAC: Authorizing `identity:get_user(user_id=09320cb2423c4c1689a3fbca9ff22266)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:54.595885 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9747aa-2c7b-4091-b22a-2ad18a761799 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:54.602451 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 85/175] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Oct 14 02:48:54 2022] GET /identity/v3/users/09320cb2423c4c1689a3fbca9ff22266 => generated 283 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:48:54.602451 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9747aa-2c7b-4091-b22a-2ad18a761799 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:54.675437 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712f2e0a-aeda-4779-bd43-b96a8652e211 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:54.680023 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:54.697414 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:54.714539 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4267e8-3262-40ee-b3ec-6e705604775d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:54.715260 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4267e8-3262-40ee-b3ec-6e705604775d None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:54.715543 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4267e8-3262-40ee-b3ec-6e705604775d None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:54.715824 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4267e8-3262-40ee-b3ec-6e705604775d None admin] PATH_INFO: `/v3/users/5fa67b25acbc40ddb6b2beb3e3933e0f` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:54.718704 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9747aa-2c7b-4091-b22a-2ad18a761799 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:54.730320 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4267e8-3262-40ee-b3ec-6e705604775d None admin] RBAC: Authorizing `identity:get_user(user_id=5fa67b25acbc40ddb6b2beb3e3933e0f)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:54.731812 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4267e8-3262-40ee-b3ec-6e705604775d None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:54.733573 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 91/176] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Oct 14 02:48:54 2022] GET /identity/v3/users/5fa67b25acbc40ddb6b2beb3e3933e0f => generated 297 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:48:54.734361 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4267e8-3262-40ee-b3ec-6e705604775d None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:54.764504 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed9e669-939f-4c8e-83f1-09de1e37d77a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:54.765115 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed9e669-939f-4c8e-83f1-09de1e37d77a None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:54.765395 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed9e669-939f-4c8e-83f1-09de1e37d77a None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:54.765666 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed9e669-939f-4c8e-83f1-09de1e37d77a None admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:54.766361 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed9e669-939f-4c8e-83f1-09de1e37d77a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:48:54.766663 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed9e669-939f-4c8e-83f1-09de1e37d77a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:54.768046 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed9e669-939f-4c8e-83f1-09de1e37d77a None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:54.776770 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 86/177] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Oct 14 02:48:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:48:54.777889 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed9e669-939f-4c8e-83f1-09de1e37d77a None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:54.857280 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4267e8-3262-40ee-b3ec-6e705604775d None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:54.893830 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fa26dc-cd46-4336-af4c-fd8b771e3735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:54.894679 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fa26dc-cd46-4336-af4c-fd8b771e3735 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:54.895133 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fa26dc-cd46-4336-af4c-fd8b771e3735 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:54.895556 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fa26dc-cd46-4336-af4c-fd8b771e3735 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:54.913438 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed9e669-939f-4c8e-83f1-09de1e37d77a None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:54.940256 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:54.960111 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20885ee8-c4ec-451b-98a6-dfac88499fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:54.961272 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20885ee8-c4ec-451b-98a6-dfac88499fde None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:54.961705 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20885ee8-c4ec-451b-98a6-dfac88499fde None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:54.962377 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20885ee8-c4ec-451b-98a6-dfac88499fde None admin] PATH_INFO: `/v3/projects/b5c28654bb91452aaa2aff4568067821` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:54.963959 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20885ee8-c4ec-451b-98a6-dfac88499fde None admin] RBAC: Authorizing `identity:get_project(project_id=b5c28654bb91452aaa2aff4568067821)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:54.965562 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20885ee8-c4ec-451b-98a6-dfac88499fde None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:54.968341 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 87/178] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Oct 14 02:48:54 2022] GET /identity/v3/projects/b5c28654bb91452aaa2aff4568067821 => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:48:54.969441 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20885ee8-c4ec-451b-98a6-dfac88499fde None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:54.983497 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fa26dc-cd46-4336-af4c-fd8b771e3735 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:54.985139 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fa26dc-cd46-4336-af4c-fd8b771e3735 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:54.986695 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:55.093290 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20885ee8-c4ec-451b-98a6-dfac88499fde None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:55.127438 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 92/179] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Oct 14 02:48:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 14 02:48:55.128692 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fa26dc-cd46-4336-af4c-fd8b771e3735 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:55.130962 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac9b94a-d8df-4a0e-94ad-250d9a0dd765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:55.132036 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac9b94a-d8df-4a0e-94ad-250d9a0dd765 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:55.132036 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac9b94a-d8df-4a0e-94ad-250d9a0dd765 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:55.132334 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac9b94a-d8df-4a0e-94ad-250d9a0dd765 None admin] PATH_INFO: `/v3/users/09320cb2423c4c1689a3fbca9ff22266` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:55.134433 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac9b94a-d8df-4a0e-94ad-250d9a0dd765 None admin] RBAC: Authorizing `identity:get_user(user_id=09320cb2423c4c1689a3fbca9ff22266)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:55.135926 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac9b94a-d8df-4a0e-94ad-250d9a0dd765 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:55.137670 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 88/180] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Oct 14 02:48:54 2022] GET /identity/v3/users/09320cb2423c4c1689a3fbca9ff22266 => generated 283 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:48:55.138327 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac9b94a-d8df-4a0e-94ad-250d9a0dd765 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:55.222255 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:55.244294 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:55.259129 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fa26dc-cd46-4336-af4c-fd8b771e3735 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:55.274164 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac9b94a-d8df-4a0e-94ad-250d9a0dd765 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:55.296990 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9371383-4832-4484-9934-fd607c8a6829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:55.297869 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9371383-4832-4484-9934-fd607c8a6829 None admin] REQUEST_METHOD: `PUT` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:55.298213 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9371383-4832-4484-9934-fd607c8a6829 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:55.298583 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9371383-4832-4484-9934-fd607c8a6829 None admin] PATH_INFO: `/v3/projects/b5c28654bb91452aaa2aff4568067821/users/5fa67b25acbc40ddb6b2beb3e3933e0f/roles/9bd0470b544240209a420f640c79c06c` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:55.300630 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9371383-4832-4484-9934-fd607c8a6829 None admin] RBAC: Authorizing `identity:create_grant(project_id=b5c28654bb91452aaa2aff4568067821, user_id=5fa67b25acbc40ddb6b2beb3e3933e0f, role_id=9bd0470b544240209a420f640c79c06c)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:55.302212 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9371383-4832-4484-9934-fd607c8a6829 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:55.315928 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae284bb-1944-4470-988e-66c5a6b3069b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:55.325926 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae284bb-1944-4470-988e-66c5a6b3069b None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:55.325926 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae284bb-1944-4470-988e-66c5a6b3069b None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:55.325926 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae284bb-1944-4470-988e-66c5a6b3069b None admin] PATH_INFO: `/v3/projects/b5c28654bb91452aaa2aff4568067821` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:55.325926 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae284bb-1944-4470-988e-66c5a6b3069b None admin] RBAC: Authorizing `identity:get_project(project_id=b5c28654bb91452aaa2aff4568067821)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:55.325926 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae284bb-1944-4470-988e-66c5a6b3069b None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:55.325926 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 89/181] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Oct 14 02:48:55 2022] GET /identity/v3/projects/b5c28654bb91452aaa2aff4568067821 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:48:55.326500 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae284bb-1944-4470-988e-66c5a6b3069b None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:55.368597 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 93/182] 10.222.0.39 () {58 vars in 1439 bytes} [Fri Oct 14 02:48:55 2022] PUT /identity/v3/projects/b5c28654bb91452aaa2aff4568067821/users/5fa67b25acbc40ddb6b2beb3e3933e0f/roles/9bd0470b544240209a420f640c79c06c => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:48:55.370051 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9371383-4832-4484-9934-fd607c8a6829 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:55.443283 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae284bb-1944-4470-988e-66c5a6b3069b None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:55.488953 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b5b440-9ff2-40ff-8821-5ff9ae8f2038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:55.489583 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b5b440-9ff2-40ff-8821-5ff9ae8f2038 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:55.489899 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b5b440-9ff2-40ff-8821-5ff9ae8f2038 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:55.490177 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b5b440-9ff2-40ff-8821-5ff9ae8f2038 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:55.495162 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9371383-4832-4484-9934-fd607c8a6829 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:55.511398 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:55.522903 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:55.535945 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49375d6-e614-4cd8-a726-5a3c08bf8e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:55.536962 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49375d6-e614-4cd8-a726-5a3c08bf8e4b None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:55.537472 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49375d6-e614-4cd8-a726-5a3c08bf8e4b None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:55.538912 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49375d6-e614-4cd8-a726-5a3c08bf8e4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:55.539870 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49375d6-e614-4cd8-a726-5a3c08bf8e4b None admin] RBAC: Adding query filter params (role.id=9e86d182c0594e0785b72ba5f390a114, scope.project.id=b5c28654bb91452aaa2aff4568067821, user.id=09320cb2423c4c1689a3fbca9ff22266) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:48:55.540382 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49375d6-e614-4cd8-a726-5a3c08bf8e4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:55.542143 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49375d6-e614-4cd8-a726-5a3c08bf8e4b None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:55.548891 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 94/183] 10.222.0.39 () {56 vars in 1374 bytes} [Fri Oct 14 02:48:55 2022] GET /identity/v3/role_assignments?user.id=09320cb2423c4c1689a3fbca9ff22266&scope.project.id=b5c28654bb91452aaa2aff4568067821&role.id=9e86d182c0594e0785b72ba5f390a114 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:48:55.581077 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b5b440-9ff2-40ff-8821-5ff9ae8f2038 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:55.582894 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b5b440-9ff2-40ff-8821-5ff9ae8f2038 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:55.703234 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 90/184] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Oct 14 02:48:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 14 02:48:55.709022 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49375d6-e614-4cd8-a726-5a3c08bf8e4b None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:55.840976 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49375d6-e614-4cd8-a726-5a3c08bf8e4b None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:55.884042 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e6f2b4-e6df-4acc-b38d-01f1a7c52e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:55.885105 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e6f2b4-e6df-4acc-b38d-01f1a7c52e33 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:55.885637 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e6f2b4-e6df-4acc-b38d-01f1a7c52e33 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:55.886236 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e6f2b4-e6df-4acc-b38d-01f1a7c52e33 None admin] PATH_INFO: `/v3/projects/b5c28654bb91452aaa2aff4568067821` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:55.887579 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e6f2b4-e6df-4acc-b38d-01f1a7c52e33 None admin] RBAC: Authorizing `identity:get_project(project_id=b5c28654bb91452aaa2aff4568067821)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:55.889298 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e6f2b4-e6df-4acc-b38d-01f1a7c52e33 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:55.891741 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 95/185] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Oct 14 02:48:55 2022] GET /identity/v3/projects/b5c28654bb91452aaa2aff4568067821 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:48:55.898009 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b5b440-9ff2-40ff-8821-5ff9ae8f2038 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:55.950165 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:56.019931 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b5b440-9ff2-40ff-8821-5ff9ae8f2038 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:56.057255 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8119d6-7659-475b-a36d-f9726f95d6a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:56.058079 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8119d6-7659-475b-a36d-f9726f95d6a3 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:56.058463 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8119d6-7659-475b-a36d-f9726f95d6a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:56.058844 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8119d6-7659-475b-a36d-f9726f95d6a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:56.059671 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8119d6-7659-475b-a36d-f9726f95d6a3 None admin] RBAC: Adding query filter params (role.id=d695dc72739e4aa7a50d5fb73ad1a91f, scope.project.id=b5c28654bb91452aaa2aff4568067821, user.id=09320cb2423c4c1689a3fbca9ff22266) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:48:56.060095 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8119d6-7659-475b-a36d-f9726f95d6a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:56.061647 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8119d6-7659-475b-a36d-f9726f95d6a3 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:56.068112 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 91/186] 10.222.0.39 () {56 vars in 1374 bytes} [Fri Oct 14 02:48:55 2022] GET /identity/v3/role_assignments?user.id=09320cb2423c4c1689a3fbca9ff22266&scope.project.id=b5c28654bb91452aaa2aff4568067821&role.id=d695dc72739e4aa7a50d5fb73ad1a91f => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:48:56.884687 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d2bd76-edcb-4b75-b5c7-802ad58c44a8 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:56.885230 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d2bd76-edcb-4b75-b5c7-802ad58c44a8 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:56.885230 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d2bd76-edcb-4b75-b5c7-802ad58c44a8 None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:56.886736 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 96/187] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:48:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:48:56.891664 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b72f6e7-ed95-4ab8-a26f-e2a35aceee85 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:56.892077 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b72f6e7-ed95-4ab8-a26f-e2a35aceee85 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:56.892237 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b72f6e7-ed95-4ab8-a26f-e2a35aceee85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:56.910158 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-7b72f6e7-ed95-4ab8-a26f-e2a35aceee85 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:48:56.911356 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-7b72f6e7-ed95-4ab8-a26f-e2a35aceee85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:48:56.980634 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 92/188] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:48:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:48:57.079560 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf1f90c-c186-4194-a5b6-1273e5ee3cb8 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:57.079863 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf1f90c-c186-4194-a5b6-1273e5ee3cb8 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:57.080114 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf1f90c-c186-4194-a5b6-1273e5ee3cb8 None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:57.081611 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 97/189] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:48:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:48:57.086802 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e1475-7405-4901-b015-90ccf3e1fcf0 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:57.087088 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e1475-7405-4901-b015-90ccf3e1fcf0 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:57.087382 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e1475-7405-4901-b015-90ccf3e1fcf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:57.105388 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-a88e1475-7405-4901-b015-90ccf3e1fcf0 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:48:57.106582 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-a88e1475-7405-4901-b015-90ccf3e1fcf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:48:57.141987 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:57.178186 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 93/190] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:48:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:48:57.244520 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf1f90c-c186-4194-a5b6-1273e5ee3cb8 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:57.300607 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:57.346571 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf1f90c-c186-4194-a5b6-1273e5ee3cb8 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:57.381242 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3556f2b-54f7-48b6-b8c9-1a051ebd5016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:57.381926 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3556f2b-54f7-48b6-b8c9-1a051ebd5016 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:57.382209 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3556f2b-54f7-48b6-b8c9-1a051ebd5016 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:57.382482 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3556f2b-54f7-48b6-b8c9-1a051ebd5016 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:57.393073 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3556f2b-54f7-48b6-b8c9-1a051ebd5016 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:48:57.398501 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-a3556f2b-54f7-48b6-b8c9-1a051ebd5016 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 14 02:48:57.398501 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:48:57.398501 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:48:57.398501 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:48:57.398501 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:48:57.398501 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:48:57.398501 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:48:57.398501 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:48:57.398501 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:48:57.398501 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:48:57.398501 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:48:57.398501 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:48:57.398501 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:48:57.398501 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:48:57.398501 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:48:57.398501 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:48:57.399355 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:48:57.399355 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:48:57.399355 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:48:57.399355 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:48:57.399355 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:48:57.399355 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:48:57.399355 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:48:57.399355 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:48:57.399355 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:48:57.399355 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:48:57.399355 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:48:57.399355 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:48:57.399355 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:48:57.399355 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:48:57.399355 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:48:57.399355 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:48:57.400150 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:48:57.400150 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:48:57.400150 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:48:57.400150 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:48:57.400150 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:48:57.400150 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:48:57.400150 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:48:57.400150 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:48:57.400150 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:48:57.400150 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:48:57.400150 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 14 02:48:57.400150 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:48:57.400150 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 98/191] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Oct 14 02:48:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:48:57.406719 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88e1475-7405-4901-b015-90ccf3e1fcf0 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:57.444158 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3556f2b-54f7-48b6-b8c9-1a051ebd5016 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:57.513686 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88e1475-7405-4901-b015-90ccf3e1fcf0 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:57.551099 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3556f2b-54f7-48b6-b8c9-1a051ebd5016 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:57.551710 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb68e2cf-6e91-43af-b7bd-f833ba94ccaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:57.552708 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb68e2cf-6e91-43af-b7bd-f833ba94ccaa None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:57.553982 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb68e2cf-6e91-43af-b7bd-f833ba94ccaa None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:57.554268 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb68e2cf-6e91-43af-b7bd-f833ba94ccaa None admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:57.554945 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb68e2cf-6e91-43af-b7bd-f833ba94ccaa None admin] RBAC: Adding query filter params (name=reader) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:48:57.555250 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb68e2cf-6e91-43af-b7bd-f833ba94ccaa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:57.556664 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb68e2cf-6e91-43af-b7bd-f833ba94ccaa None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:57.565027 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 94/192] 10.222.0.39 () {56 vars in 1101 bytes} [Fri Oct 14 02:48:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:48:57.565815 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:57.570356 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb68e2cf-6e91-43af-b7bd-f833ba94ccaa None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:57.588376 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d88921-ddc6-40f2-b967-f3b5fa9ee68d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:57.589005 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d88921-ddc6-40f2-b967-f3b5fa9ee68d None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:57.589284 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d88921-ddc6-40f2-b967-f3b5fa9ee68d None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:57.589620 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d88921-ddc6-40f2-b967-f3b5fa9ee68d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:57.591520 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d88921-ddc6-40f2-b967-f3b5fa9ee68d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:57.593476 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d88921-ddc6-40f2-b967-f3b5fa9ee68d None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:57.595237 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 99/193] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:48:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:48:57.600990 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d88921-ddc6-40f2-b967-f3b5fa9ee68d None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:57.605243 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:57.678703 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb68e2cf-6e91-43af-b7bd-f833ba94ccaa None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:57.703032 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d88921-ddc6-40f2-b967-f3b5fa9ee68d None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:57.715140 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d6216f-c971-4086-b287-f68a9686f904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:57.715777 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d6216f-c971-4086-b287-f68a9686f904 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:57.716063 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d6216f-c971-4086-b287-f68a9686f904 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:57.716346 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d6216f-c971-4086-b287-f68a9686f904 None admin] PATH_INFO: `/v3/users/19a8bf10d03d4835adb98ae9cc0a4817` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:57.733266 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d6216f-c971-4086-b287-f68a9686f904 None admin] RBAC: Authorizing `identity:get_user(user_id=19a8bf10d03d4835adb98ae9cc0a4817)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:57.734931 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d6216f-c971-4086-b287-f68a9686f904 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:57.741258 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 95/194] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Oct 14 02:48:57 2022] GET /identity/v3/users/19a8bf10d03d4835adb98ae9cc0a4817 => generated 297 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:48:57.741445 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81339820-bc2f-47cd-a208-3003384c25a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:57.741445 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81339820-bc2f-47cd-a208-3003384c25a2 None admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:57.741631 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81339820-bc2f-47cd-a208-3003384c25a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:57.741931 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81339820-bc2f-47cd-a208-3003384c25a2 None admin] PATH_INFO: `/v3/users` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:57.742463 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d6216f-c971-4086-b287-f68a9686f904 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:57.742560 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81339820-bc2f-47cd-a208-3003384c25a2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:57.744038 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81339820-bc2f-47cd-a208-3003384c25a2 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:57.813419 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 100/195] 10.222.0.39 () {60 vars in 1132 bytes} [Fri Oct 14 02:48:57 2022] POST /identity/v3/users => generated 293 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 02:48:57.842951 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:57.854998 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d6216f-c971-4086-b287-f68a9686f904 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:57.864657 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb54a67c-a249-4039-9c1a-77853aaff1d0 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:57.864991 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb54a67c-a249-4039-9c1a-77853aaff1d0 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:57.865211 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb54a67c-a249-4039-9c1a-77853aaff1d0 None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:57.866649 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 101/196] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:48:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 14 02:48:57.872424 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be8a168-db0c-472d-a564-541039a7b786 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:57.872749 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be8a168-db0c-472d-a564-541039a7b786 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:57.872984 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be8a168-db0c-472d-a564-541039a7b786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:57.892206 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-8be8a168-db0c-472d-a564-541039a7b786 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:48:57.893453 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-8be8a168-db0c-472d-a564-541039a7b786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:48:57.898257 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045534a2-d3f7-4c6b-aaa0-c8bb5c9e14fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:57.899507 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045534a2-d3f7-4c6b-aaa0-c8bb5c9e14fd None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:57.900067 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045534a2-d3f7-4c6b-aaa0-c8bb5c9e14fd None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:57.900516 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045534a2-d3f7-4c6b-aaa0-c8bb5c9e14fd None admin] PATH_INFO: `/v3/projects/b5c28654bb91452aaa2aff4568067821` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:57.901968 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045534a2-d3f7-4c6b-aaa0-c8bb5c9e14fd None admin] RBAC: Authorizing `identity:get_project(project_id=b5c28654bb91452aaa2aff4568067821)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:57.903731 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045534a2-d3f7-4c6b-aaa0-c8bb5c9e14fd None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:57.905993 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 96/197] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Oct 14 02:48:57 2022] GET /identity/v3/projects/b5c28654bb91452aaa2aff4568067821 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:48:57.912408 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045534a2-d3f7-4c6b-aaa0-c8bb5c9e14fd None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:57.931250 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:57.972312 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 102/198] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:48:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 14 02:48:58.028666 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045534a2-d3f7-4c6b-aaa0-c8bb5c9e14fd None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:58.064416 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bf0ec6-d901-492f-946d-54f7bdb55aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:58.065083 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bf0ec6-d901-492f-946d-54f7bdb55aac None admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:58.065369 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bf0ec6-d901-492f-946d-54f7bdb55aac None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:58.065648 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bf0ec6-d901-492f-946d-54f7bdb55aac None admin] PATH_INFO: `/v3/projects/b5c28654bb91452aaa2aff4568067821/users/19a8bf10d03d4835adb98ae9cc0a4817/roles/3026af19537b46cab1318d0e13ae4163` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:58.067610 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bf0ec6-d901-492f-946d-54f7bdb55aac None admin] RBAC: Authorizing `identity:create_grant(project_id=b5c28654bb91452aaa2aff4568067821, user_id=19a8bf10d03d4835adb98ae9cc0a4817, role_id=3026af19537b46cab1318d0e13ae4163)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:58.069549 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bf0ec6-d901-492f-946d-54f7bdb55aac None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:58.154070 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 97/199] 10.222.0.39 () {58 vars in 1439 bytes} [Fri Oct 14 02:48:57 2022] PUT /identity/v3/projects/b5c28654bb91452aaa2aff4568067821/users/19a8bf10d03d4835adb98ae9cc0a4817/roles/3026af19537b46cab1318d0e13ae4163 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 02:48:58.240539 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be8a168-db0c-472d-a564-541039a7b786 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:58.360798 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be8a168-db0c-472d-a564-541039a7b786 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:58.398060 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ba17ef-b284-440e-9b7f-db1becc815b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:58.398703 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ba17ef-b284-440e-9b7f-db1becc815b0 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:58.398985 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ba17ef-b284-440e-9b7f-db1becc815b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:58.399256 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ba17ef-b284-440e-9b7f-db1becc815b0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:58.410063 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ba17ef-b284-440e-9b7f-db1becc815b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:48:58.415552 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-48ba17ef-b284-440e-9b7f-db1becc815b0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 14 02:48:58.415552 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:48:58.415552 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:48:58.415552 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:48:58.415552 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:48:58.415552 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:48:58.415552 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:48:58.415552 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:48:58.415552 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:48:58.415552 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:48:58.415552 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:48:58.415552 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:48:58.415552 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:48:58.415552 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:48:58.415552 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:48:58.415552 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:48:58.416430 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:48:58.416430 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:48:58.416430 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:48:58.416430 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:48:58.416430 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:48:58.416430 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:48:58.416430 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:48:58.416430 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:48:58.416430 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:48:58.416430 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:48:58.416430 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:48:58.416430 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:48:58.416430 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:48:58.416430 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:48:58.416430 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:48:58.416430 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:48:58.417232 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:48:58.417232 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:48:58.417232 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:48:58.417232 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:48:58.417232 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:48:58.417232 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:48:58.417232 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:48:58.417232 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:48:58.417232 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:48:58.417232 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:48:58.417232 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 14 02:48:58.417232 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:48:58.417232 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 103/200] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Oct 14 02:48:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:48:58.422712 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bf0ec6-d901-492f-946d-54f7bdb55aac None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:58.441911 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:58.525888 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bf0ec6-d901-492f-946d-54f7bdb55aac None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:58.560476 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdef0c97-277a-42c5-8fdf-cfa18173ba35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:58.561094 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef0c97-277a-42c5-8fdf-cfa18173ba35 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:58.561383 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef0c97-277a-42c5-8fdf-cfa18173ba35 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:58.561658 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef0c97-277a-42c5-8fdf-cfa18173ba35 None admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:58.562364 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdef0c97-277a-42c5-8fdf-cfa18173ba35 None admin] RBAC: Adding query filter params (name=member) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:48:58.562671 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdef0c97-277a-42c5-8fdf-cfa18173ba35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:58.564055 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdef0c97-277a-42c5-8fdf-cfa18173ba35 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:58.576656 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 98/201] 10.222.0.39 () {56 vars in 1101 bytes} [Fri Oct 14 02:48:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:48:58.578538 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ba17ef-b284-440e-9b7f-db1becc815b0 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:58.609439 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:58.687767 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ba17ef-b284-440e-9b7f-db1becc815b0 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:58.725359 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a80be29-377f-4c91-89a2-3039b25e8102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:58.726053 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a80be29-377f-4c91-89a2-3039b25e8102 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:58.726340 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a80be29-377f-4c91-89a2-3039b25e8102 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:58.726614 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a80be29-377f-4c91-89a2-3039b25e8102 None admin] PATH_INFO: `/v3/users/5fa67b25acbc40ddb6b2beb3e3933e0f` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:58.727774 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a80be29-377f-4c91-89a2-3039b25e8102 None admin] RBAC: Authorizing `identity:get_user(user_id=5fa67b25acbc40ddb6b2beb3e3933e0f)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:58.729296 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a80be29-377f-4c91-89a2-3039b25e8102 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:58.731193 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 104/202] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Oct 14 02:48:58 2022] GET /identity/v3/users/5fa67b25acbc40ddb6b2beb3e3933e0f => generated 297 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:48:58.736958 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdef0c97-277a-42c5-8fdf-cfa18173ba35 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:58.820709 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:58.840243 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdef0c97-277a-42c5-8fdf-cfa18173ba35 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:58.874102 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68da4a4b-bbc5-43ea-bfc4-0685a3d5903d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:58.874718 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68da4a4b-bbc5-43ea-bfc4-0685a3d5903d None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:58.875003 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68da4a4b-bbc5-43ea-bfc4-0685a3d5903d None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:58.875277 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68da4a4b-bbc5-43ea-bfc4-0685a3d5903d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:58.949526 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68da4a4b-bbc5-43ea-bfc4-0685a3d5903d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:58.951047 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68da4a4b-bbc5-43ea-bfc4-0685a3d5903d None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:59.030629 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:59.049775 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 99/203] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Oct 14 02:48:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 14 02:48:59.056190 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a80be29-377f-4c91-89a2-3039b25e8102 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:59.163786 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:48:59.169643 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a80be29-377f-4c91-89a2-3039b25e8102 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:59.205216 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9374dd1-cf9c-4745-b286-1bb9f6561ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:59.205872 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9374dd1-cf9c-4745-b286-1bb9f6561ab2 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:59.206153 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9374dd1-cf9c-4745-b286-1bb9f6561ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:59.206433 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9374dd1-cf9c-4745-b286-1bb9f6561ab2 None admin] PATH_INFO: `/v3/projects/b5c28654bb91452aaa2aff4568067821` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:59.207611 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9374dd1-cf9c-4745-b286-1bb9f6561ab2 None admin] RBAC: Authorizing `identity:get_project(project_id=b5c28654bb91452aaa2aff4568067821)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:59.209249 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9374dd1-cf9c-4745-b286-1bb9f6561ab2 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:59.216318 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 105/204] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Oct 14 02:48:59 2022] GET /identity/v3/projects/b5c28654bb91452aaa2aff4568067821 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:48:59.217421 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68da4a4b-bbc5-43ea-bfc4-0685a3d5903d None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:48:59.320541 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68da4a4b-bbc5-43ea-bfc4-0685a3d5903d None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:48:59.354885 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bcb492-7099-454f-987b-5155b4ff7b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:48:59.355496 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bcb492-7099-454f-987b-5155b4ff7b21 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:48:59.355779 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bcb492-7099-454f-987b-5155b4ff7b21 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:48:59.356053 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bcb492-7099-454f-987b-5155b4ff7b21 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:48:59.356774 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bcb492-7099-454f-987b-5155b4ff7b21 None admin] RBAC: Adding query filter params (role.id=9bd0470b544240209a420f640c79c06c, scope.project.id=b5c28654bb91452aaa2aff4568067821, user.id=5fa67b25acbc40ddb6b2beb3e3933e0f) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:48:59.357084 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bcb492-7099-454f-987b-5155b4ff7b21 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:48:59.358568 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bcb492-7099-454f-987b-5155b4ff7b21 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:48:59.364684 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 100/205] 10.222.0.39 () {56 vars in 1374 bytes} [Fri Oct 14 02:48:59 2022] GET /identity/v3/role_assignments?user.id=5fa67b25acbc40ddb6b2beb3e3933e0f&scope.project.id=b5c28654bb91452aaa2aff4568067821&role.id=9bd0470b544240209a420f640c79c06c => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:49:00.213008 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8175246-f5a8-4c62-805b-7464b49851ab None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:00.213520 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8175246-f5a8-4c62-805b-7464b49851ab None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:00.213628 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8175246-f5a8-4c62-805b-7464b49851ab None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:00.215090 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 106/206] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:49:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:49:00.220393 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698da241-f7f0-4261-aacb-4ac3102c352f None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:00.220901 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698da241-f7f0-4261-aacb-4ac3102c352f None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:00.221073 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698da241-f7f0-4261-aacb-4ac3102c352f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:00.222373 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d18f667-5fd2-4b9a-ae33-0e4b7fab470c None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:00.222657 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d18f667-5fd2-4b9a-ae33-0e4b7fab470c None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:00.222931 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d18f667-5fd2-4b9a-ae33-0e4b7fab470c None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:00.224184 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 107/207] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:49:00 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:49:00.230290 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80046996-c5bd-4a2d-bafa-e90e089eb8cf None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:00.230571 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80046996-c5bd-4a2d-bafa-e90e089eb8cf None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:00.230841 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80046996-c5bd-4a2d-bafa-e90e089eb8cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:00.241674 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-698da241-f7f0-4261-aacb-4ac3102c352f None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:49:00.242959 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-698da241-f7f0-4261-aacb-4ac3102c352f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:49:00.249344 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-80046996-c5bd-4a2d-bafa-e90e089eb8cf None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:49:00.250559 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-80046996-c5bd-4a2d-bafa-e90e089eb8cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:49:00.284059 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:00.316573 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 101/208] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:49:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:49:00.322694 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 108/209] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:49:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:49:00.418941 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce37c69e-6ae8-4d6e-b398-438f0fe82250 None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:00.419286 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce37c69e-6ae8-4d6e-b398-438f0fe82250 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:00.419508 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce37c69e-6ae8-4d6e-b398-438f0fe82250 None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:00.420985 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 102/210] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:49:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:49:00.425961 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7ca56a-8643-4302-8132-5d7da900c07d None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:00.426253 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7ca56a-8643-4302-8132-5d7da900c07d None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:00.426531 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7ca56a-8643-4302-8132-5d7da900c07d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:00.444791 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-4f7ca56a-8643-4302-8132-5d7da900c07d None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:49:00.445984 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-4f7ca56a-8643-4302-8132-5d7da900c07d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:49:00.512091 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 109/211] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:49:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:49:00.588046 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce37c69e-6ae8-4d6e-b398-438f0fe82250 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:00.591281 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7ca56a-8643-4302-8132-5d7da900c07d None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:00.617628 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:00.697810 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7ca56a-8643-4302-8132-5d7da900c07d None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:00.700610 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce37c69e-6ae8-4d6e-b398-438f0fe82250 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:00.734405 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ee2014-00ec-472e-8a31-098f23c00e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:00.735076 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee2014-00ec-472e-8a31-098f23c00e17 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:00.735389 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee2014-00ec-472e-8a31-098f23c00e17 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:00.735620 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee2014-00ec-472e-8a31-098f23c00e17 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:00.736727 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ee2014-00ec-472e-8a31-098f23c00e17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:00.738242 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ee2014-00ec-472e-8a31-098f23c00e17 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:00.738569 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538f81aa-5b2c-4cea-a8d7-0593996badf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:00.739198 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538f81aa-5b2c-4cea-a8d7-0593996badf3 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:00.739483 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538f81aa-5b2c-4cea-a8d7-0593996badf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:00.739759 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538f81aa-5b2c-4cea-a8d7-0593996badf3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:00.739850 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 110/212] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:49:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:00.745270 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ee2014-00ec-472e-8a31-098f23c00e17 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:00.752625 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538f81aa-5b2c-4cea-a8d7-0593996badf3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:49:00.757505 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-538f81aa-5b2c-4cea-a8d7-0593996badf3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 14 02:49:00.757505 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:00.757505 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:00.757505 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:00.757505 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:00.757505 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:00.757505 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:00.757505 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:00.757505 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:00.757505 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:00.757505 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:00.757505 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:00.757505 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:49:00.757505 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:49:00.757505 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:49:00.757505 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:49:00.758517 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:49:00.758517 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:49:00.758517 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:49:00.758517 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:49:00.758517 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:00.758517 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:00.758517 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:00.758517 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:00.758517 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:00.758517 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:00.758517 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:00.758517 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:00.758517 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:00.758517 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:00.758517 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:00.758517 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:00.759408 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:00.759408 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:00.759408 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:00.759408 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:00.759408 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:49:00.759408 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:49:00.759408 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:49:00.759408 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:49:00.759408 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:49:00.759408 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:49:00.759408 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 14 02:49:00.759408 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:00.759408 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 103/213] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Oct 14 02:49:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:00.764258 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538f81aa-5b2c-4cea-a8d7-0593996badf3 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:00.800579 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:00.838405 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:00.846728 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ee2014-00ec-472e-8a31-098f23c00e17 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:00.864328 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538f81aa-5b2c-4cea-a8d7-0593996badf3 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:00.881721 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c7b5a2-9d46-41b7-a94f-4ac0feceae24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:00.882363 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c7b5a2-9d46-41b7-a94f-4ac0feceae24 None admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:00.882645 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c7b5a2-9d46-41b7-a94f-4ac0feceae24 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:00.882917 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c7b5a2-9d46-41b7-a94f-4ac0feceae24 None admin] PATH_INFO: `/v3/users` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:00.883592 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c7b5a2-9d46-41b7-a94f-4ac0feceae24 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:00.885036 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c7b5a2-9d46-41b7-a94f-4ac0feceae24 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:00.897678 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37def3ca-80d3-4461-b5c9-dfe0e50bfdd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:00.898309 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37def3ca-80d3-4461-b5c9-dfe0e50bfdd4 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:00.898584 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37def3ca-80d3-4461-b5c9-dfe0e50bfdd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:00.898850 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37def3ca-80d3-4461-b5c9-dfe0e50bfdd4 None admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:00.899502 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37def3ca-80d3-4461-b5c9-dfe0e50bfdd4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:00.899793 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37def3ca-80d3-4461-b5c9-dfe0e50bfdd4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:00.901150 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37def3ca-80d3-4461-b5c9-dfe0e50bfdd4 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:00.908927 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 104/214] 10.222.0.39 () {56 vars in 1101 bytes} [Fri Oct 14 02:49:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:49:00.909826 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37def3ca-80d3-4461-b5c9-dfe0e50bfdd4 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:00.978934 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 111/215] 10.222.0.39 () {60 vars in 1132 bytes} [Fri Oct 14 02:49:00 2022] POST /identity/v3/users => generated 293 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 02:49:00.980017 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c7b5a2-9d46-41b7-a94f-4ac0feceae24 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:01.010128 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:01.016478 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37def3ca-80d3-4461-b5c9-dfe0e50bfdd4 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:01.053837 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf44a53c-d870-4c71-8799-9182d473fc46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:01.054502 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf44a53c-d870-4c71-8799-9182d473fc46 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:01.055197 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf44a53c-d870-4c71-8799-9182d473fc46 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:01.055522 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf44a53c-d870-4c71-8799-9182d473fc46 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:01.066955 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf44a53c-d870-4c71-8799-9182d473fc46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:49:01.072921 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-bf44a53c-d870-4c71-8799-9182d473fc46 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 14 02:49:01.072921 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:01.072921 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:01.072921 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:01.072921 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:01.072921 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:01.072921 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:01.072921 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:01.072921 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:01.072921 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:01.072921 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:01.072921 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:01.072921 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:49:01.072921 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:49:01.072921 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:49:01.072921 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:49:01.073954 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:49:01.073954 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:49:01.073954 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:49:01.073954 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:49:01.073954 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:01.073954 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:01.073954 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:01.073954 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:01.073954 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:01.073954 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:01.073954 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:01.073954 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:01.073954 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:01.073954 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:01.073954 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:01.073954 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:01.074781 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:01.074781 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:01.074781 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:01.074781 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:01.074781 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:49:01.074781 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:49:01.074781 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:49:01.074781 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:49:01.074781 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:49:01.074781 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:49:01.074781 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 14 02:49:01.074781 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:01.074781 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 105/216] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Oct 14 02:49:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:01.080192 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf44a53c-d870-4c71-8799-9182d473fc46 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:01.109730 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c7b5a2-9d46-41b7-a94f-4ac0feceae24 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:01.149787 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c4de43-173d-456c-9671-43edb1d9008f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:01.150683 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c4de43-173d-456c-9671-43edb1d9008f None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:01.150972 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c4de43-173d-456c-9671-43edb1d9008f None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:01.151248 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c4de43-173d-456c-9671-43edb1d9008f None admin] PATH_INFO: `/v3/users/b1a9bfaa6e1e405280490fddf6d155f0` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:01.166664 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c4de43-173d-456c-9671-43edb1d9008f None admin] RBAC: Authorizing `identity:get_user(user_id=b1a9bfaa6e1e405280490fddf6d155f0)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:01.168504 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c4de43-173d-456c-9671-43edb1d9008f None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:01.170397 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 112/217] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Oct 14 02:49:00 2022] GET /identity/v3/users/b1a9bfaa6e1e405280490fddf6d155f0 => generated 293 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:01.176296 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c4de43-173d-456c-9671-43edb1d9008f None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:01.187365 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:01.190684 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf44a53c-d870-4c71-8799-9182d473fc46 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:01.233422 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d78686f-ef1b-4809-a165-dd193fea8e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:01.234184 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d78686f-ef1b-4809-a165-dd193fea8e3a None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:01.234412 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d78686f-ef1b-4809-a165-dd193fea8e3a None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:01.234659 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d78686f-ef1b-4809-a165-dd193fea8e3a None admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:01.235442 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d78686f-ef1b-4809-a165-dd193fea8e3a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:01.235626 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d78686f-ef1b-4809-a165-dd193fea8e3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:01.237026 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d78686f-ef1b-4809-a165-dd193fea8e3a None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:01.249336 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 106/218] 10.222.0.39 () {56 vars in 1101 bytes} [Fri Oct 14 02:49:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:49:01.255142 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d78686f-ef1b-4809-a165-dd193fea8e3a None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:01.279206 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:01.297422 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c4de43-173d-456c-9671-43edb1d9008f None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:01.343127 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64df1cf3-3b33-48c2-8376-66e229cf102a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:01.343738 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64df1cf3-3b33-48c2-8376-66e229cf102a None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:01.344017 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64df1cf3-3b33-48c2-8376-66e229cf102a None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:01.344290 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64df1cf3-3b33-48c2-8376-66e229cf102a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:01.345024 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64df1cf3-3b33-48c2-8376-66e229cf102a None admin] RBAC: Adding query filter params (role.id=9bd0470b544240209a420f640c79c06c, user.id=b1a9bfaa6e1e405280490fddf6d155f0, scope.system=all) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:01.345334 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64df1cf3-3b33-48c2-8376-66e229cf102a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:01.347154 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64df1cf3-3b33-48c2-8376-66e229cf102a None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:01.352519 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 113/219] 10.222.0.39 () {56 vars in 1308 bytes} [Fri Oct 14 02:49:01 2022] GET /identity/v3/role_assignments?user.id=b1a9bfaa6e1e405280490fddf6d155f0&scope.system=all&role.id=9bd0470b544240209a420f640c79c06c => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:01.376567 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d78686f-ef1b-4809-a165-dd193fea8e3a None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:01.416476 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97792843-5520-4194-9ea0-dc09b046cecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:01.417563 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97792843-5520-4194-9ea0-dc09b046cecc None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:01.418079 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97792843-5520-4194-9ea0-dc09b046cecc None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:01.418557 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97792843-5520-4194-9ea0-dc09b046cecc None admin] PATH_INFO: `/v3/users/19a8bf10d03d4835adb98ae9cc0a4817` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:01.420193 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97792843-5520-4194-9ea0-dc09b046cecc None admin] RBAC: Authorizing `identity:get_user(user_id=19a8bf10d03d4835adb98ae9cc0a4817)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:01.422074 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97792843-5520-4194-9ea0-dc09b046cecc None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:01.424327 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 107/220] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Oct 14 02:49:01 2022] GET /identity/v3/users/19a8bf10d03d4835adb98ae9cc0a4817 => generated 297 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:01.430383 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64df1cf3-3b33-48c2-8376-66e229cf102a None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:01.509703 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:01.544775 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64df1cf3-3b33-48c2-8376-66e229cf102a None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:01.590302 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff4dd23-a5de-4bce-ad33-e6394e0f32eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:01.590913 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff4dd23-a5de-4bce-ad33-e6394e0f32eb None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:01.591198 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff4dd23-a5de-4bce-ad33-e6394e0f32eb None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:01.591472 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff4dd23-a5de-4bce-ad33-e6394e0f32eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:01.663573 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff4dd23-a5de-4bce-ad33-e6394e0f32eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:01.665383 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff4dd23-a5de-4bce-ad33-e6394e0f32eb None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:01.738608 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:01.770363 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 114/221] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Oct 14 02:49:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 14 02:49:01.776227 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97792843-5520-4194-9ea0-dc09b046cecc None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:01.867042 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:01.882798 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97792843-5520-4194-9ea0-dc09b046cecc None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:01.917508 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ad1d4c-f244-4282-960e-0f3f1d1277b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:01.918100 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ad1d4c-f244-4282-960e-0f3f1d1277b9 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:01.918381 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ad1d4c-f244-4282-960e-0f3f1d1277b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:01.918656 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ad1d4c-f244-4282-960e-0f3f1d1277b9 None admin] PATH_INFO: `/v3/projects/b5c28654bb91452aaa2aff4568067821` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:01.919782 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ad1d4c-f244-4282-960e-0f3f1d1277b9 None admin] RBAC: Authorizing `identity:get_project(project_id=b5c28654bb91452aaa2aff4568067821)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:01.921240 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ad1d4c-f244-4282-960e-0f3f1d1277b9 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:01.923087 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 108/222] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Oct 14 02:49:01 2022] GET /identity/v3/projects/b5c28654bb91452aaa2aff4568067821 => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:49:01.931080 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff4dd23-a5de-4bce-ad33-e6394e0f32eb None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:02.043670 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff4dd23-a5de-4bce-ad33-e6394e0f32eb None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:02.086042 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1709be8-91dd-4359-9b68-87b203b84ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:02.086645 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1709be8-91dd-4359-9b68-87b203b84ed4 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:02.086921 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1709be8-91dd-4359-9b68-87b203b84ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:02.087191 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1709be8-91dd-4359-9b68-87b203b84ed4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:02.087908 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1709be8-91dd-4359-9b68-87b203b84ed4 None admin] RBAC: Adding query filter params (role.id=3026af19537b46cab1318d0e13ae4163, scope.project.id=b5c28654bb91452aaa2aff4568067821, user.id=19a8bf10d03d4835adb98ae9cc0a4817) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:02.088212 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1709be8-91dd-4359-9b68-87b203b84ed4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:02.089645 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1709be8-91dd-4359-9b68-87b203b84ed4 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:02.095814 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 115/223] 10.222.0.39 () {56 vars in 1374 bytes} [Fri Oct 14 02:49:01 2022] GET /identity/v3/role_assignments?user.id=19a8bf10d03d4835adb98ae9cc0a4817&scope.project.id=b5c28654bb91452aaa2aff4568067821&role.id=3026af19537b46cab1318d0e13ae4163 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:03.328594 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18be8f3-eca0-4890-b595-5f1770765a65 None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:03.329090 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18be8f3-eca0-4890-b595-5f1770765a65 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:03.329166 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18be8f3-eca0-4890-b595-5f1770765a65 None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:03.330673 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 109/224] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:49:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:49:03.336444 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a71e71-9284-473b-ab45-52030e555f71 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:03.336863 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a71e71-9284-473b-ab45-52030e555f71 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:03.337043 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a71e71-9284-473b-ab45-52030e555f71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:03.355267 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-02a71e71-9284-473b-ab45-52030e555f71 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:49:03.356432 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-02a71e71-9284-473b-ab45-52030e555f71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:49:03.384320 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:03.410046 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96ad951-246e-424d-b80d-4ff070b60df4 None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:03.410363 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96ad951-246e-424d-b80d-4ff070b60df4 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:03.410607 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96ad951-246e-424d-b80d-4ff070b60df4 None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:03.412204 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 110/225] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:49:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 14 02:49:03.417078 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627a2d10-1ee9-4428-8cf3-5439aabda09b None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:03.417370 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627a2d10-1ee9-4428-8cf3-5439aabda09b None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:03.417649 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627a2d10-1ee9-4428-8cf3-5439aabda09b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:03.429633 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 116/226] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:49:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:49:03.435506 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-627a2d10-1ee9-4428-8cf3-5439aabda09b None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:49:03.436708 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-627a2d10-1ee9-4428-8cf3-5439aabda09b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:49:03.509615 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 111/227] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:49:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 14 02:49:03.567807 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06fd5c8-0441-436c-82ab-b147699902b7 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:03.568165 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06fd5c8-0441-436c-82ab-b147699902b7 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:03.568513 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06fd5c8-0441-436c-82ab-b147699902b7 None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:03.570520 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 117/228] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:49:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:49:03.575564 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f40e63-3760-4069-b02e-4ea811c237be None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:03.575856 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f40e63-3760-4069-b02e-4ea811c237be None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:03.576133 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f40e63-3760-4069-b02e-4ea811c237be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:03.594305 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-12f40e63-3760-4069-b02e-4ea811c237be None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:49:03.595419 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-12f40e63-3760-4069-b02e-4ea811c237be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:49:03.662267 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 112/229] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:49:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:49:03.696096 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06fd5c8-0441-436c-82ab-b147699902b7 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:03.775308 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f40e63-3760-4069-b02e-4ea811c237be None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:03.791983 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:03.800707 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06fd5c8-0441-436c-82ab-b147699902b7 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:03.839566 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ca38f4-6b57-4a70-8990-8e36424e768a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:03.840456 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ca38f4-6b57-4a70-8990-8e36424e768a None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:03.840953 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ca38f4-6b57-4a70-8990-8e36424e768a None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:03.841228 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ca38f4-6b57-4a70-8990-8e36424e768a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:03.852446 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ca38f4-6b57-4a70-8990-8e36424e768a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:49:03.858620 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-b6ca38f4-6b57-4a70-8990-8e36424e768a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 14 02:49:03.858620 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:03.858620 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:03.858620 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:03.858620 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:03.858620 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:03.858620 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:03.858620 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:03.858620 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:03.858620 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:03.858620 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:03.858620 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:03.858620 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:49:03.858620 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:49:03.858620 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:49:03.858620 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:49:03.859487 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:49:03.859487 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:49:03.859487 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:49:03.859487 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:49:03.859487 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:03.859487 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:03.859487 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:03.859487 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:03.859487 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:03.859487 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:03.859487 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:03.859487 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:03.859487 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:03.859487 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:03.859487 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:03.859487 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:03.861059 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:03.861059 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:03.861059 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:03.861059 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:03.861059 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:49:03.861059 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:49:03.861059 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:49:03.861059 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:49:03.861059 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:49:03.861059 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:49:03.861059 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 14 02:49:03.861059 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:49:03.861059 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 118/230] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Oct 14 02:49:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:03.865299 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ca38f4-6b57-4a70-8990-8e36424e768a None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:03.881044 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f40e63-3760-4069-b02e-4ea811c237be None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:03.893961 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:03.917873 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc223a8a-23a7-4458-a3bd-9f33ba579b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:03.918506 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc223a8a-23a7-4458-a3bd-9f33ba579b67 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:03.918852 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc223a8a-23a7-4458-a3bd-9f33ba579b67 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:03.919055 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc223a8a-23a7-4458-a3bd-9f33ba579b67 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:03.920189 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc223a8a-23a7-4458-a3bd-9f33ba579b67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:03.921627 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc223a8a-23a7-4458-a3bd-9f33ba579b67 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:03.923417 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 113/231] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:49:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:49:03.928559 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc223a8a-23a7-4458-a3bd-9f33ba579b67 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:03.969028 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ca38f4-6b57-4a70-8990-8e36424e768a None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:04.003811 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eea9d1-f1b2-40b5-8469-2f600bc7a232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:04.004649 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eea9d1-f1b2-40b5-8469-2f600bc7a232 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:04.005127 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eea9d1-f1b2-40b5-8469-2f600bc7a232 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:04.005580 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eea9d1-f1b2-40b5-8469-2f600bc7a232 None admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:04.006483 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6eea9d1-f1b2-40b5-8469-2f600bc7a232 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:04.006785 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6eea9d1-f1b2-40b5-8469-2f600bc7a232 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:04.008182 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6eea9d1-f1b2-40b5-8469-2f600bc7a232 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:04.010605 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:04.016727 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 119/232] 10.222.0.39 () {56 vars in 1101 bytes} [Fri Oct 14 02:49:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:49:04.017500 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eea9d1-f1b2-40b5-8469-2f600bc7a232 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:04.029547 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc223a8a-23a7-4458-a3bd-9f33ba579b67 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:04.064116 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704105df-b6db-4f0a-b943-72e3187f96c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:04.064738 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704105df-b6db-4f0a-b943-72e3187f96c6 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:04.065014 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704105df-b6db-4f0a-b943-72e3187f96c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:04.065283 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704105df-b6db-4f0a-b943-72e3187f96c6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:04.075637 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704105df-b6db-4f0a-b943-72e3187f96c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:49:04.080371 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-704105df-b6db-4f0a-b943-72e3187f96c6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 14 02:49:04.080371 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:04.080371 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:04.080371 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:04.080371 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:04.080371 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:04.080371 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:04.080371 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:04.080371 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:04.080371 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:04.080371 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:04.080371 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:04.080371 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:49:04.080371 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:49:04.080371 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:49:04.080371 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:49:04.081241 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:49:04.081241 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:49:04.081241 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:49:04.081241 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:49:04.081241 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:04.081241 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:04.081241 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:04.081241 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:04.081241 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:04.081241 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:04.081241 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:04.081241 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:04.081241 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:04.081241 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:04.081241 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:04.081241 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:04.086754 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:04.086754 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:04.086754 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:04.086754 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:04.086754 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:49:04.086754 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:49:04.086754 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:49:04.086754 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:49:04.086754 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:49:04.086754 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:49:04.086754 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 14 02:49:04.086754 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:04.086754 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 114/233] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Oct 14 02:49:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 02:49:04.086754 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704105df-b6db-4f0a-b943-72e3187f96c6 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:04.102101 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:04.117676 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eea9d1-f1b2-40b5-8469-2f600bc7a232 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:04.150662 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e02379-6758-4711-ae8f-d8b36071fd33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:04.151266 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e02379-6758-4711-ae8f-d8b36071fd33 None admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:04.151553 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e02379-6758-4711-ae8f-d8b36071fd33 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:04.151829 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e02379-6758-4711-ae8f-d8b36071fd33 None admin] PATH_INFO: `/v3/groups` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:04.152456 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e02379-6758-4711-ae8f-d8b36071fd33 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:04.153932 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e02379-6758-4711-ae8f-d8b36071fd33 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:04.183173 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704105df-b6db-4f0a-b943-72e3187f96c6 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:04.213102 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 120/234] 10.222.0.39 () {60 vars in 1134 bytes} [Fri Oct 14 02:49:04 2022] POST /identity/v3/groups => generated 229 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:49:04.214039 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e02379-6758-4711-ae8f-d8b36071fd33 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:04.215534 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2fb118-e54b-4ee1-bf89-d0388cbe8684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:04.216177 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2fb118-e54b-4ee1-bf89-d0388cbe8684 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:04.216512 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2fb118-e54b-4ee1-bf89-d0388cbe8684 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:04.216790 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2fb118-e54b-4ee1-bf89-d0388cbe8684 None admin] PATH_INFO: `/v3/users/7bc593e33c744b02a7675265558c34f3` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:04.230417 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:04.233218 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2fb118-e54b-4ee1-bf89-d0388cbe8684 None admin] RBAC: Authorizing `identity:get_user(user_id=7bc593e33c744b02a7675265558c34f3)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:04.234777 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2fb118-e54b-4ee1-bf89-d0388cbe8684 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:04.240270 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 115/235] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Oct 14 02:49:04 2022] GET /identity/v3/users/7bc593e33c744b02a7675265558c34f3 => generated 293 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:04.243107 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2fb118-e54b-4ee1-bf89-d0388cbe8684 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:04.355460 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2fb118-e54b-4ee1-bf89-d0388cbe8684 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:04.359117 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e02379-6758-4711-ae8f-d8b36071fd33 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:04.393881 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1601708c-3810-4714-ac32-678cab26c98f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:04.394409 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1601708c-3810-4714-ac32-678cab26c98f None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:04.394693 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1601708c-3810-4714-ac32-678cab26c98f None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:04.394969 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1601708c-3810-4714-ac32-678cab26c98f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:04.395696 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1601708c-3810-4714-ac32-678cab26c98f None admin] RBAC: Adding query filter params (role.id=3026af19537b46cab1318d0e13ae4163, user.id=7bc593e33c744b02a7675265558c34f3, scope.system=all) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:04.396005 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1601708c-3810-4714-ac32-678cab26c98f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:04.397968 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1601708c-3810-4714-ac32-678cab26c98f None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:04.399771 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e97e85-267b-46c7-83ac-e99a85e1eaf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:04.399771 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e97e85-267b-46c7-83ac-e99a85e1eaf0 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:04.400061 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e97e85-267b-46c7-83ac-e99a85e1eaf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:04.400430 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e97e85-267b-46c7-83ac-e99a85e1eaf0 None admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:04.401104 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e97e85-267b-46c7-83ac-e99a85e1eaf0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:04.401414 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e97e85-267b-46c7-83ac-e99a85e1eaf0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:04.402876 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e97e85-267b-46c7-83ac-e99a85e1eaf0 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:04.403005 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 116/236] 10.222.0.39 () {56 vars in 1308 bytes} [Fri Oct 14 02:49:04 2022] GET /identity/v3/role_assignments?user.id=7bc593e33c744b02a7675265558c34f3&scope.system=all&role.id=3026af19537b46cab1318d0e13ae4163 => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:49:04.412132 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 121/237] 10.222.0.39 () {56 vars in 1101 bytes} [Fri Oct 14 02:49:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:04.418719 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1601708c-3810-4714-ac32-678cab26c98f None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:04.492621 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:04.532347 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1601708c-3810-4714-ac32-678cab26c98f None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:04.569258 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee48e66-ae54-4d8c-a0e1-5fe8202062c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:04.569936 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee48e66-ae54-4d8c-a0e1-5fe8202062c8 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:04.570217 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee48e66-ae54-4d8c-a0e1-5fe8202062c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:04.570487 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee48e66-ae54-4d8c-a0e1-5fe8202062c8 None admin] PATH_INFO: `/v3/users/b1a9bfaa6e1e405280490fddf6d155f0` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:04.571616 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee48e66-ae54-4d8c-a0e1-5fe8202062c8 None admin] RBAC: Authorizing `identity:get_user(user_id=b1a9bfaa6e1e405280490fddf6d155f0)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:04.573054 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee48e66-ae54-4d8c-a0e1-5fe8202062c8 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:04.574843 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 117/238] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Oct 14 02:49:04 2022] GET /identity/v3/users/b1a9bfaa6e1e405280490fddf6d155f0 => generated 293 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:04.580256 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e97e85-267b-46c7-83ac-e99a85e1eaf0 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:04.593093 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:04.685689 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e97e85-267b-46c7-83ac-e99a85e1eaf0 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:04.718362 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0274719-b323-4fef-ac7e-6d1b7c28f476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:04.719011 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0274719-b323-4fef-ac7e-6d1b7c28f476 None admin] REQUEST_METHOD: `PUT` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:04.719294 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0274719-b323-4fef-ac7e-6d1b7c28f476 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:04.719567 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0274719-b323-4fef-ac7e-6d1b7c28f476 None admin] PATH_INFO: `/v3/system/users/b1a9bfaa6e1e405280490fddf6d155f0/roles/9bd0470b544240209a420f640c79c06c` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:04.720978 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0274719-b323-4fef-ac7e-6d1b7c28f476 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=b1a9bfaa6e1e405280490fddf6d155f0, role_id=9bd0470b544240209a420f640c79c06c)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:04.722405 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0274719-b323-4fef-ac7e-6d1b7c28f476 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:04.802728 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 122/239] 10.222.0.39 () {58 vars in 1334 bytes} [Fri Oct 14 02:49:04 2022] PUT /identity/v3/system/users/b1a9bfaa6e1e405280490fddf6d155f0/roles/9bd0470b544240209a420f640c79c06c => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:49:06.409161 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc0e30d-a931-45ea-8f8b-6c9990433bdf None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:06.409673 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc0e30d-a931-45ea-8f8b-6c9990433bdf None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:06.409749 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc0e30d-a931-45ea-8f8b-6c9990433bdf None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:06.411119 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 118/240] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:49:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:49:06.417067 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f20d9d-a680-4893-b75c-705aef1c990c None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:06.417551 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f20d9d-a680-4893-b75c-705aef1c990c None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:06.417622 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f20d9d-a680-4893-b75c-705aef1c990c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:06.437667 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-65f20d9d-a680-4893-b75c-705aef1c990c None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:49:06.438936 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-65f20d9d-a680-4893-b75c-705aef1c990c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:49:06.509393 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 123/241] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:49:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:49:06.545081 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7a4179-f0e8-4559-9331-ac01758caee2 None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:06.545377 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7a4179-f0e8-4559-9331-ac01758caee2 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:06.545642 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7a4179-f0e8-4559-9331-ac01758caee2 None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:06.547146 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 119/242] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:49:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:49:06.552052 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549268eb-4e0f-47bc-b697-a24323f148c7 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:06.552347 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549268eb-4e0f-47bc-b697-a24323f148c7 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:06.552620 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549268eb-4e0f-47bc-b697-a24323f148c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:06.558934 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:06.571491 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-549268eb-4e0f-47bc-b697-a24323f148c7 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:49:06.572799 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-549268eb-4e0f-47bc-b697-a24323f148c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:49:06.640273 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 124/243] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:49:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:49:06.779064 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7a4179-f0e8-4559-9331-ac01758caee2 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:06.885019 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7a4179-f0e8-4559-9331-ac01758caee2 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:06.911400 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549268eb-4e0f-47bc-b697-a24323f148c7 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:06.918163 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:06.923830 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e2e588-3515-49dc-bc4d-167eb05d134b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:06.924477 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e2e588-3515-49dc-bc4d-167eb05d134b None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:06.924756 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e2e588-3515-49dc-bc4d-167eb05d134b None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:06.925029 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e2e588-3515-49dc-bc4d-167eb05d134b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:06.926170 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e2e588-3515-49dc-bc4d-167eb05d134b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:06.941547 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e2e588-3515-49dc-bc4d-167eb05d134b None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:06.941547 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 120/244] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:49:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:06.941547 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98371211-6451-4a79-b427-12c68135eb9f None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:06.941547 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98371211-6451-4a79-b427-12c68135eb9f None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:06.941547 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98371211-6451-4a79-b427-12c68135eb9f None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:06.941547 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 121/245] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:49:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:49:06.941547 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98371211-6451-4a79-b427-12c68135eb9f None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:07.014601 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549268eb-4e0f-47bc-b697-a24323f148c7 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:07.036038 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98371211-6451-4a79-b427-12c68135eb9f None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:07.043881 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:07.048420 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f06559f-efc1-4be7-bfd1-a79da6660079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:07.049046 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f06559f-efc1-4be7-bfd1-a79da6660079 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:07.049323 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f06559f-efc1-4be7-bfd1-a79da6660079 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:07.049597 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f06559f-efc1-4be7-bfd1-a79da6660079 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:07.059361 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f06559f-efc1-4be7-bfd1-a79da6660079 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:49:07.064334 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-8f06559f-efc1-4be7-bfd1-a79da6660079 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 14 02:49:07.064334 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:07.064334 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:07.064334 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:07.064334 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:07.064334 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:07.064334 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:07.064334 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:07.064334 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:07.064334 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:07.064334 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:07.064334 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:07.064334 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:49:07.064334 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:49:07.064334 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:49:07.064334 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:49:07.065221 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:49:07.065221 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:49:07.065221 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:49:07.065221 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:49:07.065221 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:07.065221 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:07.065221 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:07.065221 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:07.065221 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:07.065221 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:07.065221 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:07.065221 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:07.065221 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:07.065221 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:07.065221 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:07.065221 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:07.070432 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:07.070432 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:07.070432 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:07.070432 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:07.070432 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:49:07.070432 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:49:07.070432 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:49:07.070432 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:49:07.070432 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:49:07.070432 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:49:07.070432 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 14 02:49:07.070432 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:49:07.070432 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 125/246] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Oct 14 02:49:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 02:49:07.070432 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b9871-ff51-403d-8b21-18b5918367ad None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:07.071148 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b9871-ff51-403d-8b21-18b5918367ad None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:07.071148 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b9871-ff51-403d-8b21-18b5918367ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:07.071275 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d9023c-f5f8-4637-8db0-e8d30ac42be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:07.071275 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d9023c-f5f8-4637-8db0-e8d30ac42be0 None admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:07.071275 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d9023c-f5f8-4637-8db0-e8d30ac42be0 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:07.071275 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d9023c-f5f8-4637-8db0-e8d30ac42be0 None admin] PATH_INFO: `/v3/groups` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:07.071791 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d9023c-f5f8-4637-8db0-e8d30ac42be0 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:07.073505 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d9023c-f5f8-4637-8db0-e8d30ac42be0 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:07.087099 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-1b9b9871-ff51-403d-8b21-18b5918367ad None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:49:07.088283 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-1b9b9871-ff51-403d-8b21-18b5918367ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:49:07.152511 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 126/247] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:49:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:49:07.153313 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9b9871-ff51-403d-8b21-18b5918367ad None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:07.229802 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 122/248] 10.222.0.39 () {60 vars in 1134 bytes} [Fri Oct 14 02:49:06 2022] POST /identity/v3/groups => generated 226 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:49:07.267497 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9b9871-ff51-403d-8b21-18b5918367ad None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:07.293940 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:07.306458 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebc15ec-df55-48db-aff2-6f3934348026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:07.309429 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebc15ec-df55-48db-aff2-6f3934348026 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:07.309429 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebc15ec-df55-48db-aff2-6f3934348026 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:07.309429 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebc15ec-df55-48db-aff2-6f3934348026 None admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:07.309429 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebc15ec-df55-48db-aff2-6f3934348026 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:07.309429 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebc15ec-df55-48db-aff2-6f3934348026 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:07.310338 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebc15ec-df55-48db-aff2-6f3934348026 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:07.321026 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 127/249] 10.222.0.39 () {56 vars in 1101 bytes} [Fri Oct 14 02:49:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:07.326892 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d9023c-f5f8-4637-8db0-e8d30ac42be0 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:07.396257 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:07.419495 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebc15ec-df55-48db-aff2-6f3934348026 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:07.445982 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d9023c-f5f8-4637-8db0-e8d30ac42be0 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:07.484142 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d259059-965c-4e51-85a7-e55522408b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:07.484782 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d259059-965c-4e51-85a7-e55522408b98 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:07.485062 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d259059-965c-4e51-85a7-e55522408b98 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:07.485335 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d259059-965c-4e51-85a7-e55522408b98 None admin] PATH_INFO: `/v3/users/7bc593e33c744b02a7675265558c34f3` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:07.486488 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d259059-965c-4e51-85a7-e55522408b98 None admin] RBAC: Authorizing `identity:get_user(user_id=7bc593e33c744b02a7675265558c34f3)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:07.487964 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d259059-965c-4e51-85a7-e55522408b98 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:07.489559 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 123/250] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Oct 14 02:49:07 2022] GET /identity/v3/users/7bc593e33c744b02a7675265558c34f3 => generated 293 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:49:07.496161 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d259059-965c-4e51-85a7-e55522408b98 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:07.545396 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebc15ec-df55-48db-aff2-6f3934348026 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:07.587830 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8342d3e5-a25c-4a25-872d-f4d69478637b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:07.589050 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8342d3e5-a25c-4a25-872d-f4d69478637b None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:07.589424 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8342d3e5-a25c-4a25-872d-f4d69478637b None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:07.589647 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8342d3e5-a25c-4a25-872d-f4d69478637b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:07.604800 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8342d3e5-a25c-4a25-872d-f4d69478637b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:49:07.609945 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-8342d3e5-a25c-4a25-872d-f4d69478637b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 14 02:49:07.609945 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:07.609945 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:07.609945 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:07.609945 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:07.609945 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:07.609945 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:07.609945 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:07.609945 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:07.609945 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:07.609945 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:07.609945 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:07.609945 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:49:07.609945 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:49:07.609945 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:49:07.609945 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:49:07.610827 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:49:07.610827 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:49:07.610827 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:49:07.610827 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:49:07.610827 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:07.610827 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:07.610827 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:07.610827 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:07.610827 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:07.610827 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:07.610827 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:07.610827 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:07.610827 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:07.610827 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:07.610827 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:07.610827 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:07.611618 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:07.611618 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:07.611618 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:07.611618 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:07.611618 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:49:07.611618 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:49:07.611618 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:49:07.611618 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:49:07.611618 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:49:07.611618 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:49:07.611618 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 14 02:49:07.611618 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:49:07.611618 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 128/251] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Oct 14 02:49:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 02:49:07.619495 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8342d3e5-a25c-4a25-872d-f4d69478637b None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:07.621922 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d259059-965c-4e51-85a7-e55522408b98 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:07.659184 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:07.670244 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddae0a3-10f9-4e62-b71b-63aaafdcdbfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:07.671216 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddae0a3-10f9-4e62-b71b-63aaafdcdbfa None admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:07.671922 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddae0a3-10f9-4e62-b71b-63aaafdcdbfa None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:07.671922 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddae0a3-10f9-4e62-b71b-63aaafdcdbfa None admin] PATH_INFO: `/v3/system/users/7bc593e33c744b02a7675265558c34f3/roles/3026af19537b46cab1318d0e13ae4163` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:07.674058 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddae0a3-10f9-4e62-b71b-63aaafdcdbfa None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=7bc593e33c744b02a7675265558c34f3, role_id=3026af19537b46cab1318d0e13ae4163)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:07.675496 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddae0a3-10f9-4e62-b71b-63aaafdcdbfa None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:07.675923 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:07.733951 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8342d3e5-a25c-4a25-872d-f4d69478637b None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:07.773053 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd247e0-16fe-453a-bb01-ea4841b15d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:07.773674 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd247e0-16fe-453a-bb01-ea4841b15d89 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:07.774028 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd247e0-16fe-453a-bb01-ea4841b15d89 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:07.774325 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd247e0-16fe-453a-bb01-ea4841b15d89 None admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:07.774421 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 124/252] 10.222.0.39 () {58 vars in 1334 bytes} [Fri Oct 14 02:49:07 2022] PUT /identity/v3/system/users/7bc593e33c744b02a7675265558c34f3/roles/3026af19537b46cab1318d0e13ae4163 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 02:49:07.775016 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd247e0-16fe-453a-bb01-ea4841b15d89 None admin] RBAC: Adding query filter params (name=member) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:07.775280 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd247e0-16fe-453a-bb01-ea4841b15d89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:07.776703 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd247e0-16fe-453a-bb01-ea4841b15d89 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:07.786812 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 129/253] 10.222.0.39 () {56 vars in 1101 bytes} [Fri Oct 14 02:49:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:07.793214 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddae0a3-10f9-4e62-b71b-63aaafdcdbfa None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:07.912534 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddae0a3-10f9-4e62-b71b-63aaafdcdbfa None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:07.950117 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9d5bb4-7702-4b9b-8e45-3d5a8434c1ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:07.950877 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9d5bb4-7702-4b9b-8e45-3d5a8434c1ff None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:07.951165 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9d5bb4-7702-4b9b-8e45-3d5a8434c1ff None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:07.951486 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9d5bb4-7702-4b9b-8e45-3d5a8434c1ff None admin] PATH_INFO: `/v3/users/b1a9bfaa6e1e405280490fddf6d155f0` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:07.952678 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9d5bb4-7702-4b9b-8e45-3d5a8434c1ff None admin] RBAC: Authorizing `identity:get_user(user_id=b1a9bfaa6e1e405280490fddf6d155f0)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:07.954204 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9d5bb4-7702-4b9b-8e45-3d5a8434c1ff None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:07.956185 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 125/254] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Oct 14 02:49:07 2022] GET /identity/v3/users/b1a9bfaa6e1e405280490fddf6d155f0 => generated 293 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:07.962596 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd247e0-16fe-453a-bb01-ea4841b15d89 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:08.087904 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd247e0-16fe-453a-bb01-ea4841b15d89 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:08.097696 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:08.126659 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f98e18-97fd-4a30-a004-4a003d6b5bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:08.127631 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f98e18-97fd-4a30-a004-4a003d6b5bb9 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:08.128112 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f98e18-97fd-4a30-a004-4a003d6b5bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:08.128582 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f98e18-97fd-4a30-a004-4a003d6b5bb9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:08.129505 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f98e18-97fd-4a30-a004-4a003d6b5bb9 None admin] RBAC: Adding query filter params (role.id=9bd0470b544240209a420f640c79c06c, user.id=b1a9bfaa6e1e405280490fddf6d155f0, scope.system=all) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:08.130154 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f98e18-97fd-4a30-a004-4a003d6b5bb9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:08.132686 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f98e18-97fd-4a30-a004-4a003d6b5bb9 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:08.147253 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 130/255] 10.222.0.39 () {56 vars in 1308 bytes} [Fri Oct 14 02:49:07 2022] GET /identity/v3/role_assignments?user.id=b1a9bfaa6e1e405280490fddf6d155f0&scope.system=all&role.id=9bd0470b544240209a420f640c79c06c => generated 510 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:09.838388 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc9a9fa-ff14-49f8-b418-ad5666dbdc8a None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:09.838923 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc9a9fa-ff14-49f8-b418-ad5666dbdc8a None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:09.838923 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc9a9fa-ff14-49f8-b418-ad5666dbdc8a None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:09.840522 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 126/256] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:49:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:49:09.845556 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d0803-8b1a-4cbf-b5ea-9b27f86ec2f2 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:09.846415 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d0803-8b1a-4cbf-b5ea-9b27f86ec2f2 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:09.846883 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d0803-8b1a-4cbf-b5ea-9b27f86ec2f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:09.871704 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-132d0803-8b1a-4cbf-b5ea-9b27f86ec2f2 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:49:09.873215 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-132d0803-8b1a-4cbf-b5ea-9b27f86ec2f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:49:09.947972 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 131/257] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:49:09 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 14 02:49:09.974681 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d86e45-59b1-4c5f-bb61-260dc0ddb374 None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:09.975285 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d86e45-59b1-4c5f-bb61-260dc0ddb374 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:09.975776 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d86e45-59b1-4c5f-bb61-260dc0ddb374 None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:09.977334 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 127/258] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:49:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:49:09.983004 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6330649c-760b-4217-9fd7-8b83942e7cba None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:09.983295 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6330649c-760b-4217-9fd7-8b83942e7cba None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:09.983576 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6330649c-760b-4217-9fd7-8b83942e7cba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:10.006287 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-6330649c-760b-4217-9fd7-8b83942e7cba None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:49:10.007497 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-6330649c-760b-4217-9fd7-8b83942e7cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:49:10.024202 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 132/259] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:49:09 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 14 02:49:10.486336 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d86e45-59b1-4c5f-bb61-260dc0ddb374 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:10.503334 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d86e45-59b1-4c5f-bb61-260dc0ddb374 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:10.508238 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7a9670-358f-41f1-b0b5-dcc499b00adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:10.509098 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7a9670-358f-41f1-b0b5-dcc499b00adc admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:10.509576 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7a9670-358f-41f1-b0b5-dcc499b00adc admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:10.510066 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7a9670-358f-41f1-b0b5-dcc499b00adc admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:10.525926 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7a9670-358f-41f1-b0b5-dcc499b00adc admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:49:10.531079 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-6e7a9670-358f-41f1-b0b5-dcc499b00adc admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 14 02:49:10.531079 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:10.531079 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:10.531079 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:10.531079 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:10.531079 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:10.531079 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:10.531079 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:10.531079 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:10.531079 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:10.531079 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:10.531079 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:10.531079 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:49:10.531079 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:49:10.531079 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:49:10.531079 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:49:10.531925 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:49:10.531925 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:49:10.531925 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:49:10.531925 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:49:10.531925 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:10.531925 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:10.531925 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:10.531925 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:10.531925 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:10.531925 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:10.531925 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:10.531925 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:10.531925 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:10.531925 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:10.531925 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:10.531925 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:10.532713 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:10.532713 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:10.532713 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:10.532713 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:10.532713 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:49:10.532713 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:49:10.532713 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:49:10.532713 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:49:10.532713 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:49:10.532713 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:49:10.532713 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 14 02:49:10.532713 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:10.542118 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 128/260] 10.222.0.39 () {56 vars in 1135 bytes} [Fri Oct 14 02:49:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 57 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:10.548037 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6330649c-760b-4217-9fd7-8b83942e7cba None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:10.560299 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7a9670-358f-41f1-b0b5-dcc499b00adc admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:10.564075 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6330649c-760b-4217-9fd7-8b83942e7cba None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:10.573700 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7a9670-358f-41f1-b0b5-dcc499b00adc admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:10.576647 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809fb457-2454-4128-aeac-0b81ba6bb8e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:10.576647 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809fb457-2454-4128-aeac-0b81ba6bb8e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:10.576647 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809fb457-2454-4128-aeac-0b81ba6bb8e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:10.576647 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809fb457-2454-4128-aeac-0b81ba6bb8e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:10.577028 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809fb457-2454-4128-aeac-0b81ba6bb8e4 admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:10.577028 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809fb457-2454-4128-aeac-0b81ba6bb8e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:10.579343 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: WARNING py.warnings [None req-809fb457-2454-4128-aeac-0b81ba6bb8e4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 14 02:49:10.579343 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: warnings.warn(msg) Oct 14 02:49:10.579343 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: Oct 14 02:49:10.580512 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809fb457-2454-4128-aeac-0b81ba6bb8e4 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:10.581693 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0a21c1-06ef-47f7-a88c-9ae5cef04500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:10.582542 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0a21c1-06ef-47f7-a88c-9ae5cef04500 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:10.582985 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0a21c1-06ef-47f7-a88c-9ae5cef04500 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:10.583399 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0a21c1-06ef-47f7-a88c-9ae5cef04500 admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:10.592876 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 133/261] 10.222.0.39 () {56 vars in 1132 bytes} [Fri Oct 14 02:49:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:10.596368 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f917792a-7069-40e4-8223-6bdb11e652d7 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:10.596368 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f917792a-7069-40e4-8223-6bdb11e652d7 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:10.596368 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f917792a-7069-40e4-8223-6bdb11e652d7 None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:10.596368 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 134/262] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:49:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:49:10.599116 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f917792a-7069-40e4-8223-6bdb11e652d7 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:10.606405 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0a21c1-06ef-47f7-a88c-9ae5cef04500 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:49:10.611251 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-4d0a21c1-06ef-47f7-a88c-9ae5cef04500 admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 14 02:49:10.611251 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:10.611251 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:10.611251 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:10.611251 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:10.611251 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:10.611251 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:10.611251 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:10.611251 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:10.611251 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:10.611251 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:10.611251 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:10.611251 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:49:10.611251 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:49:10.611251 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:49:10.611251 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:49:10.612116 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:49:10.612116 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:49:10.612116 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:49:10.612116 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:49:10.612116 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:10.612116 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:10.612116 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:10.612116 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:10.612116 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:10.612116 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:10.612116 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:10.612116 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:10.612116 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:10.612116 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:10.612116 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:10.612116 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:10.613424 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:10.613424 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:10.613424 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:10.613424 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:10.613424 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:49:10.613424 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:49:10.613424 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:49:10.613424 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:49:10.613424 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:49:10.613424 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:49:10.613424 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 14 02:49:10.613424 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:10.613424 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 129/263] 10.222.0.39 () {56 vars in 1120 bytes} [Fri Oct 14 02:49:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 53 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 02:49:10.614155 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c8a5f8-a94b-4b51-a6ac-372a69617d8d None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:10.614219 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c8a5f8-a94b-4b51-a6ac-372a69617d8d None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:10.614497 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c8a5f8-a94b-4b51-a6ac-372a69617d8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:10.633316 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f917792a-7069-40e4-8223-6bdb11e652d7 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:10.640128 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-19c8a5f8-a94b-4b51-a6ac-372a69617d8d None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:49:10.641580 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-19c8a5f8-a94b-4b51-a6ac-372a69617d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:49:10.643535 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7ff28f-1a4c-4ed4-a073-9c02bd69b13b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:10.644316 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7ff28f-1a4c-4ed4-a073-9c02bd69b13b admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:10.644592 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7ff28f-1a4c-4ed4-a073-9c02bd69b13b admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:10.644980 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7ff28f-1a4c-4ed4-a073-9c02bd69b13b admin admin] PATH_INFO: `/v3/groups/5903843ac1194f018ce5e748d711cd35` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:10.657541 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7ff28f-1a4c-4ed4-a073-9c02bd69b13b admin admin] RBAC: Authorizing `identity:get_group(group_id=5903843ac1194f018ce5e748d711cd35)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:10.658750 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: WARNING py.warnings [None req-9d7ff28f-1a4c-4ed4-a073-9c02bd69b13b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 14 02:49:10.658750 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: warnings.warn(msg) Oct 14 02:49:10.658750 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: Oct 14 02:49:10.660088 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7ff28f-1a4c-4ed4-a073-9c02bd69b13b admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:10.662282 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 135/264] 10.222.0.39 () {56 vars in 1201 bytes} [Fri Oct 14 02:49:10 2022] GET /identity/v3/groups/5903843ac1194f018ce5e748d711cd35 => generated 226 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:10.663214 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7ff28f-1a4c-4ed4-a073-9c02bd69b13b admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:10.675703 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:10.684213 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7ff28f-1a4c-4ed4-a073-9c02bd69b13b admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:10.696074 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ec2a15-5784-420e-9d25-881653c77f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:10.698680 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ec2a15-5784-420e-9d25-881653c77f7d admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:10.698680 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ec2a15-5784-420e-9d25-881653c77f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:10.698680 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ec2a15-5784-420e-9d25-881653c77f7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:10.698680 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ec2a15-5784-420e-9d25-881653c77f7d admin admin] RBAC: Adding query filter params (name=member) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:10.698680 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ec2a15-5784-420e-9d25-881653c77f7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:10.702958 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ec2a15-5784-420e-9d25-881653c77f7d admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:10.720209 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 136/265] 10.222.0.39 () {56 vars in 1122 bytes} [Fri Oct 14 02:49:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:10.721074 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ec2a15-5784-420e-9d25-881653c77f7d admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:10.721365 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 130/266] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:49:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:49:10.727041 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c8a5f8-a94b-4b51-a6ac-372a69617d8d None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:10.740477 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c8a5f8-a94b-4b51-a6ac-372a69617d8d None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:10.741019 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ec2a15-5784-420e-9d25-881653c77f7d admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:10.746608 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d36c391-53e6-459f-9f29-db8ac1df86c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:10.747646 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d36c391-53e6-459f-9f29-db8ac1df86c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:10.748180 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d36c391-53e6-459f-9f29-db8ac1df86c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:10.748632 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d36c391-53e6-459f-9f29-db8ac1df86c6 admin admin] PATH_INFO: `/v3/groups/5903843ac1194f018ce5e748d711cd35` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:10.750050 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d36c391-53e6-459f-9f29-db8ac1df86c6 admin admin] RBAC: Authorizing `identity:get_group(group_id=5903843ac1194f018ce5e748d711cd35)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:10.751765 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e85c00a-bf74-4f42-a794-3f6e513273ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:10.752384 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e85c00a-bf74-4f42-a794-3f6e513273ac admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:10.752664 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e85c00a-bf74-4f42-a794-3f6e513273ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:10.752932 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e85c00a-bf74-4f42-a794-3f6e513273ac admin admin] PATH_INFO: `/v3/projects/0b15574d2808402ca6940d47f10e2db8` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:10.753305 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: WARNING py.warnings [None req-1d36c391-53e6-459f-9f29-db8ac1df86c6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 14 02:49:10.753305 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: warnings.warn(msg) Oct 14 02:49:10.753305 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: Oct 14 02:49:10.755174 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d36c391-53e6-459f-9f29-db8ac1df86c6 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:10.757991 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 131/267] 10.222.0.39 () {56 vars in 1201 bytes} [Fri Oct 14 02:49:10 2022] GET /identity/v3/groups/5903843ac1194f018ce5e748d711cd35 => generated 226 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:10.758360 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e85c00a-bf74-4f42-a794-3f6e513273ac admin admin] RBAC: Authorizing `identity:get_project(project_id=0b15574d2808402ca6940d47f10e2db8)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:10.760613 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e85c00a-bf74-4f42-a794-3f6e513273ac admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:10.762167 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 137/268] 10.222.0.39 () {56 vars in 1207 bytes} [Fri Oct 14 02:49:10 2022] GET /identity/v3/projects/0b15574d2808402ca6940d47f10e2db8 => generated 298 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:10.767505 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d36c391-53e6-459f-9f29-db8ac1df86c6 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:10.768433 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e85c00a-bf74-4f42-a794-3f6e513273ac admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:10.778551 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d36c391-53e6-459f-9f29-db8ac1df86c6 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:10.784113 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d80581e-75cc-41f5-8d26-9722ad30790f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:10.785035 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d80581e-75cc-41f5-8d26-9722ad30790f admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:10.785561 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d80581e-75cc-41f5-8d26-9722ad30790f admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:10.785964 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d80581e-75cc-41f5-8d26-9722ad30790f admin admin] PATH_INFO: `/v3/projects/0b15574d2808402ca6940d47f10e2db8` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:10.787207 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e85c00a-bf74-4f42-a794-3f6e513273ac admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:10.787455 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d80581e-75cc-41f5-8d26-9722ad30790f admin admin] RBAC: Authorizing `identity:get_project(project_id=0b15574d2808402ca6940d47f10e2db8)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:10.789911 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d80581e-75cc-41f5-8d26-9722ad30790f admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:10.792578 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 132/269] 10.222.0.39 () {56 vars in 1207 bytes} [Fri Oct 14 02:49:10 2022] GET /identity/v3/projects/0b15574d2808402ca6940d47f10e2db8 => generated 298 bytes in 25 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:10.797361 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee242704-20bd-4dd3-8804-b34a01748825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:10.798487 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee242704-20bd-4dd3-8804-b34a01748825 admin admin] REQUEST_METHOD: `PUT` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:10.798909 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee242704-20bd-4dd3-8804-b34a01748825 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:10.799019 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d80581e-75cc-41f5-8d26-9722ad30790f admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:10.799409 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee242704-20bd-4dd3-8804-b34a01748825 admin admin] PATH_INFO: `/v3/projects/0b15574d2808402ca6940d47f10e2db8/groups/5903843ac1194f018ce5e748d711cd35/roles/d695dc72739e4aa7a50d5fb73ad1a91f` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:10.804342 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee242704-20bd-4dd3-8804-b34a01748825 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b15574d2808402ca6940d47f10e2db8, group_id=5903843ac1194f018ce5e748d711cd35, role_id=d695dc72739e4aa7a50d5fb73ad1a91f)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:10.805673 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: WARNING py.warnings [None req-ee242704-20bd-4dd3-8804-b34a01748825 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 14 02:49:10.805673 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: warnings.warn(msg) Oct 14 02:49:10.805673 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: Oct 14 02:49:10.808097 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee242704-20bd-4dd3-8804-b34a01748825 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:10.810892 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d80581e-75cc-41f5-8d26-9722ad30790f admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:10.820733 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a7510c-cd6e-4174-85bf-e824777461eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:10.821817 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a7510c-cd6e-4174-85bf-e824777461eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:10.822468 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a7510c-cd6e-4174-85bf-e824777461eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:10.822990 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a7510c-cd6e-4174-85bf-e824777461eb admin admin] PATH_INFO: `/v3/projects/0b15574d2808402ca6940d47f10e2db8/groups/5903843ac1194f018ce5e748d711cd35/roles/9bd0470b544240209a420f640c79c06c` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:10.825293 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a7510c-cd6e-4174-85bf-e824777461eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b15574d2808402ca6940d47f10e2db8, group_id=5903843ac1194f018ce5e748d711cd35, role_id=9bd0470b544240209a420f640c79c06c)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:10.826655 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: WARNING py.warnings [None req-a4a7510c-cd6e-4174-85bf-e824777461eb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 14 02:49:10.826655 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: warnings.warn(msg) Oct 14 02:49:10.826655 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: Oct 14 02:49:10.829213 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a7510c-cd6e-4174-85bf-e824777461eb admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:10.910881 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 138/270] 10.222.0.39 () {58 vars in 1463 bytes} [Fri Oct 14 02:49:10 2022] PUT /identity/v3/projects/0b15574d2808402ca6940d47f10e2db8/groups/5903843ac1194f018ce5e748d711cd35/roles/d695dc72739e4aa7a50d5fb73ad1a91f => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 02:49:10.912224 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0637bfc-b028-471b-8870-c880c69a99b9 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:10.912535 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0637bfc-b028-471b-8870-c880c69a99b9 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:10.912773 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0637bfc-b028-471b-8870-c880c69a99b9 None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:10.914223 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 139/271] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:49:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:49:10.921283 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0247b37c-dd72-4ea9-abdc-7fe7f3df4300 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:10.929724 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0247b37c-dd72-4ea9-abdc-7fe7f3df4300 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:10.929724 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0247b37c-dd72-4ea9-abdc-7fe7f3df4300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:10.929933 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 133/272] 10.222.0.39 () {58 vars in 1463 bytes} [Fri Oct 14 02:49:10 2022] PUT /identity/v3/projects/0b15574d2808402ca6940d47f10e2db8/groups/5903843ac1194f018ce5e748d711cd35/roles/9bd0470b544240209a420f640c79c06c => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 02:49:10.950869 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-0247b37c-dd72-4ea9-abdc-7fe7f3df4300 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:49:10.952102 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-0247b37c-dd72-4ea9-abdc-7fe7f3df4300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:49:11.041746 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 140/273] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:49:10 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 14 02:49:11.086661 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a7510c-cd6e-4174-85bf-e824777461eb admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:11.275082 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a7510c-cd6e-4174-85bf-e824777461eb admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:11.322936 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc61c50e-a6e3-40fe-a2a3-26843bb73857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:11.323539 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc61c50e-a6e3-40fe-a2a3-26843bb73857 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:11.323817 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc61c50e-a6e3-40fe-a2a3-26843bb73857 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:11.324090 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc61c50e-a6e3-40fe-a2a3-26843bb73857 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:11.338450 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc61c50e-a6e3-40fe-a2a3-26843bb73857 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:49:11.343682 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-cc61c50e-a6e3-40fe-a2a3-26843bb73857 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 14 02:49:11.343682 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:11.343682 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:11.343682 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:11.343682 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:11.343682 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:11.343682 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:11.343682 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:11.343682 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:11.343682 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:11.343682 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:11.343682 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:11.343682 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:49:11.343682 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:49:11.343682 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:49:11.343682 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:49:11.344568 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:49:11.344568 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:49:11.344568 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:49:11.344568 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:49:11.344568 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:11.344568 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:11.344568 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:11.344568 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:11.344568 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:11.344568 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:11.344568 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:11.344568 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:11.344568 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:11.344568 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:11.344568 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:11.344568 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:11.345381 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:11.345381 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:11.345381 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:11.345381 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:11.345381 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:49:11.345381 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:49:11.345381 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:49:11.345381 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:49:11.345381 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:49:11.345381 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:49:11.345381 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 14 02:49:11.345381 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:11.345381 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 134/274] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Oct 14 02:49:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:11.352277 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0247b37c-dd72-4ea9-abdc-7fe7f3df4300 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:11.361292 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:11.463961 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0247b37c-dd72-4ea9-abdc-7fe7f3df4300 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:11.501928 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc61c50e-a6e3-40fe-a2a3-26843bb73857 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:11.503271 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fbdba1-2fce-443f-9b05-b72b6039dd31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:11.504056 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fbdba1-2fce-443f-9b05-b72b6039dd31 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:11.504480 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fbdba1-2fce-443f-9b05-b72b6039dd31 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:11.504904 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fbdba1-2fce-443f-9b05-b72b6039dd31 None admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:11.505876 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fbdba1-2fce-443f-9b05-b72b6039dd31 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:11.506323 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fbdba1-2fce-443f-9b05-b72b6039dd31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:11.507877 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fbdba1-2fce-443f-9b05-b72b6039dd31 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:11.522893 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc61c50e-a6e3-40fe-a2a3-26843bb73857 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:11.525363 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 141/275] 10.222.0.39 () {56 vars in 1101 bytes} [Fri Oct 14 02:49:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:11.531397 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fbdba1-2fce-443f-9b05-b72b6039dd31 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:11.532164 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefcad10-fc05-4f0a-8d37-6840185cfe4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:11.533074 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefcad10-fc05-4f0a-8d37-6840185cfe4d admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:11.533456 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefcad10-fc05-4f0a-8d37-6840185cfe4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:11.533868 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefcad10-fc05-4f0a-8d37-6840185cfe4d admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:11.536246 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:11.560004 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefcad10-fc05-4f0a-8d37-6840185cfe4d admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:49:11.567646 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-aefcad10-fc05-4f0a-8d37-6840185cfe4d admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 14 02:49:11.567646 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:11.567646 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:11.567646 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:11.567646 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:11.567646 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:11.567646 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:11.567646 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:11.567646 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:11.567646 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:11.567646 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:11.567646 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:11.567646 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:49:11.567646 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:49:11.567646 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:49:11.567646 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:49:11.575063 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:49:11.575063 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:49:11.575063 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:49:11.575063 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:49:11.575063 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:11.575063 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:11.575063 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:11.575063 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:11.575063 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:11.575063 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:11.575063 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:11.575063 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:11.575063 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:11.575063 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:11.575063 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:11.575063 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:11.575848 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:11.575848 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:11.575848 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:11.575848 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:11.575848 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:49:11.575848 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:49:11.575848 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:49:11.575848 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:49:11.575848 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:49:11.575848 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:49:11.575848 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 14 02:49:11.575848 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:11.579592 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 135/276] 10.222.0.39 () {56 vars in 1120 bytes} [Fri Oct 14 02:49:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 78 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:11.590321 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefcad10-fc05-4f0a-8d37-6840185cfe4d admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:11.604204 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefcad10-fc05-4f0a-8d37-6840185cfe4d admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:11.612563 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07d64e4-94ab-482b-9ebe-2dd6b57e6892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:11.613470 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d64e4-94ab-482b-9ebe-2dd6b57e6892 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:11.614937 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d64e4-94ab-482b-9ebe-2dd6b57e6892 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:11.616039 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d64e4-94ab-482b-9ebe-2dd6b57e6892 admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:11.616935 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07d64e4-94ab-482b-9ebe-2dd6b57e6892 admin admin] RBAC: Adding query filter params (name=member) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:11.617436 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07d64e4-94ab-482b-9ebe-2dd6b57e6892 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:11.618671 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: WARNING py.warnings [None req-f07d64e4-94ab-482b-9ebe-2dd6b57e6892 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 14 02:49:11.618671 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: warnings.warn(msg) Oct 14 02:49:11.618671 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: Oct 14 02:49:11.620496 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07d64e4-94ab-482b-9ebe-2dd6b57e6892 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:11.635797 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 136/277] 10.222.0.39 () {56 vars in 1122 bytes} [Fri Oct 14 02:49:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:11.644994 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07d64e4-94ab-482b-9ebe-2dd6b57e6892 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:11.658808 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07d64e4-94ab-482b-9ebe-2dd6b57e6892 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:11.668158 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab358b72-5ebd-40fb-81a5-c3bf1ce35d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:11.670968 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab358b72-5ebd-40fb-81a5-c3bf1ce35d81 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:11.671473 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab358b72-5ebd-40fb-81a5-c3bf1ce35d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:11.673661 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab358b72-5ebd-40fb-81a5-c3bf1ce35d81 admin admin] PATH_INFO: `/v3/groups/5903843ac1194f018ce5e748d711cd35` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:11.675109 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab358b72-5ebd-40fb-81a5-c3bf1ce35d81 admin admin] RBAC: Authorizing `identity:get_group(group_id=5903843ac1194f018ce5e748d711cd35)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:11.678531 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:11.683139 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab358b72-5ebd-40fb-81a5-c3bf1ce35d81 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:11.687463 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 137/278] 10.222.0.39 () {56 vars in 1201 bytes} [Fri Oct 14 02:49:11 2022] GET /identity/v3/groups/5903843ac1194f018ce5e748d711cd35 => generated 226 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:49:11.693280 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab358b72-5ebd-40fb-81a5-c3bf1ce35d81 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:11.716204 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab358b72-5ebd-40fb-81a5-c3bf1ce35d81 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:11.727228 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49367d7-54d7-4639-9f23-89739e74afc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:11.728260 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49367d7-54d7-4639-9f23-89739e74afc2 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:11.728540 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49367d7-54d7-4639-9f23-89739e74afc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:11.728812 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49367d7-54d7-4639-9f23-89739e74afc2 admin admin] PATH_INFO: `/v3/projects/b5c28654bb91452aaa2aff4568067821` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:11.733191 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49367d7-54d7-4639-9f23-89739e74afc2 admin admin] RBAC: Authorizing `identity:get_project(project_id=b5c28654bb91452aaa2aff4568067821)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:11.738817 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49367d7-54d7-4639-9f23-89739e74afc2 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:11.741555 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 138/279] 10.222.0.39 () {56 vars in 1207 bytes} [Fri Oct 14 02:49:11 2022] GET /identity/v3/projects/b5c28654bb91452aaa2aff4568067821 => generated 302 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:11.747137 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49367d7-54d7-4639-9f23-89739e74afc2 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:11.763153 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49367d7-54d7-4639-9f23-89739e74afc2 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:11.774184 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6390255-93e6-44c7-be1d-edd86a0f81e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:11.774927 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6390255-93e6-44c7-be1d-edd86a0f81e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:11.775207 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6390255-93e6-44c7-be1d-edd86a0f81e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:11.775835 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6390255-93e6-44c7-be1d-edd86a0f81e4 admin admin] PATH_INFO: `/v3/projects/b5c28654bb91452aaa2aff4568067821/groups/5903843ac1194f018ce5e748d711cd35/roles/9bd0470b544240209a420f640c79c06c` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:11.780549 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6390255-93e6-44c7-be1d-edd86a0f81e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5c28654bb91452aaa2aff4568067821, group_id=5903843ac1194f018ce5e748d711cd35, role_id=9bd0470b544240209a420f640c79c06c)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:11.783583 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6390255-93e6-44c7-be1d-edd86a0f81e4 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:11.791807 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fbdba1-2fce-443f-9b05-b72b6039dd31 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:11.833530 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8ceac7-ecba-4ebe-82ff-73d1d4b703c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:11.834733 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8ceac7-ecba-4ebe-82ff-73d1d4b703c0 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:11.835021 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8ceac7-ecba-4ebe-82ff-73d1d4b703c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:11.835299 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8ceac7-ecba-4ebe-82ff-73d1d4b703c0 None admin] PATH_INFO: `/v3/users/7bc593e33c744b02a7675265558c34f3` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:11.840386 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8ceac7-ecba-4ebe-82ff-73d1d4b703c0 None admin] RBAC: Authorizing `identity:get_user(user_id=7bc593e33c744b02a7675265558c34f3)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:11.841891 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8ceac7-ecba-4ebe-82ff-73d1d4b703c0 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:11.843681 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 142/280] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Oct 14 02:49:11 2022] GET /identity/v3/users/7bc593e33c744b02a7675265558c34f3 => generated 293 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:11.851310 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8ceac7-ecba-4ebe-82ff-73d1d4b703c0 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:11.928148 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 139/281] 10.222.0.39 () {58 vars in 1463 bytes} [Fri Oct 14 02:49:11 2022] PUT /identity/v3/projects/b5c28654bb91452aaa2aff4568067821/groups/5903843ac1194f018ce5e748d711cd35/roles/9bd0470b544240209a420f640c79c06c => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 02:49:11.929424 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbb8673-b2b9-4401-940c-3aedb4461098 None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:11.929710 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbb8673-b2b9-4401-940c-3aedb4461098 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:11.930014 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbb8673-b2b9-4401-940c-3aedb4461098 None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:11.931419 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 140/282] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:49:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:49:11.937554 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc064bc-eb95-403c-83a6-d3d12bc50e1f None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:11.937904 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc064bc-eb95-403c-83a6-d3d12bc50e1f None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:11.938176 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc064bc-eb95-403c-83a6-d3d12bc50e1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:11.984393 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-efc064bc-eb95-403c-83a6-d3d12bc50e1f None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:49:11.990425 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-efc064bc-eb95-403c-83a6-d3d12bc50e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:49:12.004989 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8ceac7-ecba-4ebe-82ff-73d1d4b703c0 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:12.069457 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b2410b-c921-4c37-9f6b-862378ef1258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:12.070032 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b2410b-c921-4c37-9f6b-862378ef1258 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:12.070317 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b2410b-c921-4c37-9f6b-862378ef1258 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:12.070590 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b2410b-c921-4c37-9f6b-862378ef1258 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:12.071456 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b2410b-c921-4c37-9f6b-862378ef1258 None admin] RBAC: Adding query filter params (role.id=3026af19537b46cab1318d0e13ae4163, user.id=7bc593e33c744b02a7675265558c34f3, scope.system=all) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:12.071739 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b2410b-c921-4c37-9f6b-862378ef1258 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:12.073397 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b2410b-c921-4c37-9f6b-862378ef1258 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:12.079022 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:12.091114 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 143/283] 10.222.0.39 () {56 vars in 1308 bytes} [Fri Oct 14 02:49:11 2022] GET /identity/v3/role_assignments?user.id=7bc593e33c744b02a7675265558c34f3&scope.system=all&role.id=3026af19537b46cab1318d0e13ae4163 => generated 510 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:49:12.101021 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 141/284] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:49:11 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 14 02:49:12.659004 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b2410b-c921-4c37-9f6b-862378ef1258 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:12.670037 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b2410b-c921-4c37-9f6b-862378ef1258 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:12.674549 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265f5ef7-de4f-4f5c-8af7-aff343ff60a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:12.675188 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265f5ef7-de4f-4f5c-8af7-aff343ff60a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:12.675464 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265f5ef7-de4f-4f5c-8af7-aff343ff60a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:12.675732 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265f5ef7-de4f-4f5c-8af7-aff343ff60a7 admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:12.691864 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265f5ef7-de4f-4f5c-8af7-aff343ff60a7 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:49:12.697394 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-265f5ef7-de4f-4f5c-8af7-aff343ff60a7 admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 14 02:49:12.697394 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:12.697394 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:12.697394 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:12.697394 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:12.697394 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:12.697394 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:12.697394 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:12.697394 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:12.697394 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:12.697394 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:12.697394 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:12.697394 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:49:12.697394 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:49:12.697394 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:49:12.697394 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:49:12.698314 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:49:12.698314 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:49:12.698314 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:49:12.698314 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:49:12.698314 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:12.698314 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:12.698314 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:12.698314 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:12.698314 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:12.698314 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:12.698314 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:12.698314 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:12.698314 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:12.698314 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:12.698314 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:12.698314 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:12.699077 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:12.699077 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:12.699077 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:12.699077 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:12.699077 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:49:12.699077 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:49:12.699077 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:49:12.699077 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:49:12.699077 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:49:12.699077 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:49:12.699077 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 14 02:49:12.699077 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:49:12.701877 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 144/285] 10.222.0.39 () {56 vars in 1135 bytes} [Fri Oct 14 02:49:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 40 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:12.705017 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc064bc-eb95-403c-83a6-d3d12bc50e1f None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:12.719226 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc064bc-eb95-403c-83a6-d3d12bc50e1f None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:12.728706 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cd0a71-8132-448e-89d3-9c70e48d2b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:12.729332 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cd0a71-8132-448e-89d3-9c70e48d2b44 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:12.729892 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cd0a71-8132-448e-89d3-9c70e48d2b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:12.730167 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cd0a71-8132-448e-89d3-9c70e48d2b44 admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:12.730848 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cd0a71-8132-448e-89d3-9c70e48d2b44 admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:12.731156 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cd0a71-8132-448e-89d3-9c70e48d2b44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:12.732764 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cd0a71-8132-448e-89d3-9c70e48d2b44 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:12.750222 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 142/286] 10.222.0.39 () {56 vars in 1132 bytes} [Fri Oct 14 02:49:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:12.756491 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265f5ef7-de4f-4f5c-8af7-aff343ff60a7 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:12.770724 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265f5ef7-de4f-4f5c-8af7-aff343ff60a7 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:12.780632 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4233eb1f-f506-4168-a28c-97eab4eb48e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:12.781322 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4233eb1f-f506-4168-a28c-97eab4eb48e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:12.781639 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4233eb1f-f506-4168-a28c-97eab4eb48e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:12.781946 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4233eb1f-f506-4168-a28c-97eab4eb48e8 admin admin] PATH_INFO: `/v3/groups/5903843ac1194f018ce5e748d711cd35` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:12.783681 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4233eb1f-f506-4168-a28c-97eab4eb48e8 admin admin] RBAC: Authorizing `identity:get_group(group_id=5903843ac1194f018ce5e748d711cd35)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:12.785661 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4233eb1f-f506-4168-a28c-97eab4eb48e8 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:12.787359 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 145/287] 10.222.0.39 () {56 vars in 1201 bytes} [Fri Oct 14 02:49:12 2022] GET /identity/v3/groups/5903843ac1194f018ce5e748d711cd35 => generated 226 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:12.793042 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cd0a71-8132-448e-89d3-9c70e48d2b44 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:12.804565 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cd0a71-8132-448e-89d3-9c70e48d2b44 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:12.813444 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f46939-264b-4dcc-ba56-26f7008fe9c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:12.814138 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f46939-264b-4dcc-ba56-26f7008fe9c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:12.814419 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f46939-264b-4dcc-ba56-26f7008fe9c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:12.814695 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f46939-264b-4dcc-ba56-26f7008fe9c7 admin admin] PATH_INFO: `/v3/projects/b5c28654bb91452aaa2aff4568067821` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:12.819220 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f46939-264b-4dcc-ba56-26f7008fe9c7 admin admin] RBAC: Authorizing `identity:get_project(project_id=b5c28654bb91452aaa2aff4568067821)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:12.821367 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f46939-264b-4dcc-ba56-26f7008fe9c7 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:12.823214 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 143/288] 10.222.0.39 () {56 vars in 1207 bytes} [Fri Oct 14 02:49:12 2022] GET /identity/v3/projects/b5c28654bb91452aaa2aff4568067821 => generated 302 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:12.829373 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4233eb1f-f506-4168-a28c-97eab4eb48e8 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:12.843399 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4233eb1f-f506-4168-a28c-97eab4eb48e8 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:12.853345 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5a7c77-77a1-4890-8abe-e9717648c075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:12.854105 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5a7c77-77a1-4890-8abe-e9717648c075 admin admin] REQUEST_METHOD: `PUT` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:12.854634 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5a7c77-77a1-4890-8abe-e9717648c075 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:12.854952 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5a7c77-77a1-4890-8abe-e9717648c075 admin admin] PATH_INFO: `/v3/projects/b5c28654bb91452aaa2aff4568067821/groups/5903843ac1194f018ce5e748d711cd35/roles/d695dc72739e4aa7a50d5fb73ad1a91f` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:12.857986 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5a7c77-77a1-4890-8abe-e9717648c075 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5c28654bb91452aaa2aff4568067821, group_id=5903843ac1194f018ce5e748d711cd35, role_id=d695dc72739e4aa7a50d5fb73ad1a91f)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:12.860781 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5a7c77-77a1-4890-8abe-e9717648c075 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:12.950662 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 146/289] 10.222.0.39 () {58 vars in 1463 bytes} [Fri Oct 14 02:49:12 2022] PUT /identity/v3/projects/b5c28654bb91452aaa2aff4568067821/groups/5903843ac1194f018ce5e748d711cd35/roles/d695dc72739e4aa7a50d5fb73ad1a91f => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:49:13.024950 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8026d8e7-0690-43d1-b76d-d58d4b6c3c57 None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:13.025611 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8026d8e7-0690-43d1-b76d-d58d4b6c3c57 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:13.025611 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8026d8e7-0690-43d1-b76d-d58d4b6c3c57 None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:13.026899 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 144/290] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:49:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:49:13.032725 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32803377-d0a5-43af-858b-2fd29473dc51 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:13.033239 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32803377-d0a5-43af-858b-2fd29473dc51 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:13.033389 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32803377-d0a5-43af-858b-2fd29473dc51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:13.059499 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-32803377-d0a5-43af-858b-2fd29473dc51 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:49:13.060753 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-32803377-d0a5-43af-858b-2fd29473dc51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:49:13.154001 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 147/291] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:49:13 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 14 02:49:13.464811 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8026d8e7-0690-43d1-b76d-d58d4b6c3c57 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:13.478898 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8026d8e7-0690-43d1-b76d-d58d4b6c3c57 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:13.485321 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fb9945-66b0-4826-bc7d-eb3552e6dd8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:13.486315 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fb9945-66b0-4826-bc7d-eb3552e6dd8a admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:13.486807 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fb9945-66b0-4826-bc7d-eb3552e6dd8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:13.487271 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fb9945-66b0-4826-bc7d-eb3552e6dd8a admin admin] PATH_INFO: `/v3/roles/admin` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:13.501182 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fb9945-66b0-4826-bc7d-eb3552e6dd8a admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:49:13.506420 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-f7fb9945-66b0-4826-bc7d-eb3552e6dd8a admin admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:49:13.506420 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:13.506420 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:13.506420 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:13.506420 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:13.506420 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:13.506420 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:13.506420 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:13.506420 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:13.506420 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:13.506420 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:13.506420 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:13.506420 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:49:13.506420 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:49:13.506420 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:49:13.506420 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:49:13.507267 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:49:13.507267 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:49:13.507267 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:49:13.507267 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:49:13.507267 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:13.507267 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:13.507267 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:13.507267 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:13.507267 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:13.507267 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:13.507267 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:13.507267 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:13.507267 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:13.507267 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:13.507267 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:13.507267 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:13.508074 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:13.508074 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:13.508074 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:13.508074 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:13.508074 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:49:13.508074 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:49:13.508074 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:49:13.508074 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:49:13.508074 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:49:13.508074 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:49:13.508074 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:49:13.508074 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:13.514170 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 145/292] 10.222.0.39 () {56 vars in 1117 bytes} [Fri Oct 14 02:49:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 46 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:13.516959 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32803377-d0a5-43af-858b-2fd29473dc51 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:13.529388 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32803377-d0a5-43af-858b-2fd29473dc51 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:13.538503 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a77ac5-7f95-46dd-9230-8c66be382a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:13.539124 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a77ac5-7f95-46dd-9230-8c66be382a70 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:13.539399 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a77ac5-7f95-46dd-9230-8c66be382a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:13.539673 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a77ac5-7f95-46dd-9230-8c66be382a70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:13.540356 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a77ac5-7f95-46dd-9230-8c66be382a70 admin admin] RBAC: Adding query filter params (name=admin) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:13.540664 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a77ac5-7f95-46dd-9230-8c66be382a70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:13.542319 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a77ac5-7f95-46dd-9230-8c66be382a70 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:13.558102 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 148/293] 10.222.0.39 () {56 vars in 1120 bytes} [Fri Oct 14 02:49:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:13.564189 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fb9945-66b0-4826-bc7d-eb3552e6dd8a admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:13.574892 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fb9945-66b0-4826-bc7d-eb3552e6dd8a admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:13.581949 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46456406-1d8a-40b2-b951-e384bdd7ad98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:13.582885 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46456406-1d8a-40b2-b951-e384bdd7ad98 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:13.583356 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46456406-1d8a-40b2-b951-e384bdd7ad98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:13.584521 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46456406-1d8a-40b2-b951-e384bdd7ad98 admin admin] PATH_INFO: `/v3/groups/d0fa86df5fa747f38b9fd14e7fa9e33b` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:13.591444 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46456406-1d8a-40b2-b951-e384bdd7ad98 admin admin] RBAC: Authorizing `identity:get_group(group_id=d0fa86df5fa747f38b9fd14e7fa9e33b)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:13.593678 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46456406-1d8a-40b2-b951-e384bdd7ad98 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:13.601468 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 146/294] 10.222.0.39 () {56 vars in 1201 bytes} [Fri Oct 14 02:49:13 2022] GET /identity/v3/groups/d0fa86df5fa747f38b9fd14e7fa9e33b => generated 229 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:13.603000 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a77ac5-7f95-46dd-9230-8c66be382a70 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:13.618830 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a77ac5-7f95-46dd-9230-8c66be382a70 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:13.630464 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b817d504-7a17-45fc-adcf-357fcc9e94a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:13.631638 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b817d504-7a17-45fc-adcf-357fcc9e94a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:13.631922 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b817d504-7a17-45fc-adcf-357fcc9e94a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:13.632195 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b817d504-7a17-45fc-adcf-357fcc9e94a9 admin admin] PATH_INFO: `/v3/projects/e86df8ec1d224299a71e492e2b3fb74d` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:13.633444 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b817d504-7a17-45fc-adcf-357fcc9e94a9 admin admin] RBAC: Authorizing `identity:get_project(project_id=e86df8ec1d224299a71e492e2b3fb74d)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:13.635300 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b817d504-7a17-45fc-adcf-357fcc9e94a9 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:13.636947 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 149/295] 10.222.0.39 () {56 vars in 1207 bytes} [Fri Oct 14 02:49:13 2022] GET /identity/v3/projects/e86df8ec1d224299a71e492e2b3fb74d => generated 344 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:13.643206 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46456406-1d8a-40b2-b951-e384bdd7ad98 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:13.657565 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46456406-1d8a-40b2-b951-e384bdd7ad98 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:13.663688 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe46ba6-0f56-43b5-8245-6219aede571c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:13.664645 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe46ba6-0f56-43b5-8245-6219aede571c admin admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:13.665120 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe46ba6-0f56-43b5-8245-6219aede571c admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:13.665689 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe46ba6-0f56-43b5-8245-6219aede571c admin admin] PATH_INFO: `/v3/projects/e86df8ec1d224299a71e492e2b3fb74d/groups/d0fa86df5fa747f38b9fd14e7fa9e33b/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:13.669038 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe46ba6-0f56-43b5-8245-6219aede571c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e86df8ec1d224299a71e492e2b3fb74d, group_id=d0fa86df5fa747f38b9fd14e7fa9e33b, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:13.672102 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe46ba6-0f56-43b5-8245-6219aede571c admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:13.774823 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 147/296] 10.222.0.39 () {58 vars in 1463 bytes} [Fri Oct 14 02:49:13 2022] PUT /identity/v3/projects/e86df8ec1d224299a71e492e2b3fb74d/groups/d0fa86df5fa747f38b9fd14e7fa9e33b/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:49:14.004818 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60fe32e-e9d0-4f42-afdc-68ea75b0b8b1 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:14.005186 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60fe32e-e9d0-4f42-afdc-68ea75b0b8b1 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:14.005388 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60fe32e-e9d0-4f42-afdc-68ea75b0b8b1 None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:14.007374 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 150/297] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:49:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:49:14.014670 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f895f3-cb3e-4927-86ff-0dc28eefbde5 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:14.014946 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f895f3-cb3e-4927-86ff-0dc28eefbde5 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:14.015226 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f895f3-cb3e-4927-86ff-0dc28eefbde5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:14.043055 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-09f895f3-cb3e-4927-86ff-0dc28eefbde5 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:49:14.044191 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-09f895f3-cb3e-4927-86ff-0dc28eefbde5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:49:14.115912 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:14.124629 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 148/298] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:49:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:49:14.247931 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc7488-2918-4e5d-b68f-b9e0623d1f4e None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:14.247931 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc7488-2918-4e5d-b68f-b9e0623d1f4e None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:14.247931 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc7488-2918-4e5d-b68f-b9e0623d1f4e None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:14.249266 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 151/299] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:49:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:49:14.254648 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c67460-06d2-4199-8cce-38d1d5ae417d None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:14.254935 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c67460-06d2-4199-8cce-38d1d5ae417d None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:14.255213 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c67460-06d2-4199-8cce-38d1d5ae417d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:14.276759 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-c5c67460-06d2-4199-8cce-38d1d5ae417d None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:49:14.278730 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-c5c67460-06d2-4199-8cce-38d1d5ae417d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:49:14.359996 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 149/300] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:49:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:49:14.582220 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befc7488-2918-4e5d-b68f-b9e0623d1f4e None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:14.698630 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:14.703332 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befc7488-2918-4e5d-b68f-b9e0623d1f4e None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:14.752005 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12c7d1b-bda2-4cd7-81ad-7fd084246d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:14.752632 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12c7d1b-bda2-4cd7-81ad-7fd084246d9d None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:14.753087 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12c7d1b-bda2-4cd7-81ad-7fd084246d9d None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:14.753368 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12c7d1b-bda2-4cd7-81ad-7fd084246d9d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:14.765719 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12c7d1b-bda2-4cd7-81ad-7fd084246d9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:49:14.768121 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c67460-06d2-4199-8cce-38d1d5ae417d None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:14.770920 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-c12c7d1b-bda2-4cd7-81ad-7fd084246d9d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 14 02:49:14.770920 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:14.770920 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:14.770920 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:14.770920 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:14.770920 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:14.770920 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:14.770920 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:14.770920 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:14.770920 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:14.770920 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:14.770920 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:14.770920 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:49:14.770920 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:49:14.770920 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:49:14.770920 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:49:14.772438 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:49:14.772438 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:49:14.772438 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:49:14.772438 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:49:14.772438 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:14.772438 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:14.772438 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:14.772438 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:14.772438 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:14.772438 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:14.772438 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:14.772438 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:14.772438 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:14.772438 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:14.772438 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:14.772438 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:14.773220 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:14.773220 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:14.773220 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:14.773220 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:14.773220 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:49:14.773220 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:49:14.773220 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:49:14.773220 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:49:14.773220 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:49:14.773220 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:49:14.773220 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 14 02:49:14.773220 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:49:14.774877 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 152/301] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Oct 14 02:49:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:14.783547 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12c7d1b-bda2-4cd7-81ad-7fd084246d9d None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:14.952641 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c67460-06d2-4199-8cce-38d1d5ae417d None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:14.960757 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12c7d1b-bda2-4cd7-81ad-7fd084246d9d None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:15.000158 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:15.022193 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c330b3b-c7ac-4779-8293-5b13aa822eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:15.023742 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c330b3b-c7ac-4779-8293-5b13aa822eef None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:15.024074 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c330b3b-c7ac-4779-8293-5b13aa822eef None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:15.024346 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c330b3b-c7ac-4779-8293-5b13aa822eef None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:15.033621 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41323dec-e94c-461d-9a06-36d6794d0bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:15.034296 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41323dec-e94c-461d-9a06-36d6794d0bcb None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:15.034575 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41323dec-e94c-461d-9a06-36d6794d0bcb None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:15.034861 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41323dec-e94c-461d-9a06-36d6794d0bcb None admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:15.035550 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41323dec-e94c-461d-9a06-36d6794d0bcb None admin] RBAC: Adding query filter params (name=member) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:15.035857 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41323dec-e94c-461d-9a06-36d6794d0bcb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:15.038861 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41323dec-e94c-461d-9a06-36d6794d0bcb None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:15.046650 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c330b3b-c7ac-4779-8293-5b13aa822eef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:49:15.051613 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 153/302] 10.222.0.39 () {56 vars in 1101 bytes} [Fri Oct 14 02:49:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:49:15.051847 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-2c330b3b-c7ac-4779-8293-5b13aa822eef None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 14 02:49:15.051847 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:15.051847 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:15.051847 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:15.051847 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:15.051847 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:15.051847 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:15.051847 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:15.051847 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:15.051847 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:15.051847 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:15.051847 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:15.051847 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:49:15.051847 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:49:15.051847 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:49:15.051847 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:49:15.052751 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:49:15.052751 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:49:15.052751 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:49:15.052751 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:49:15.052751 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:15.052751 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:15.052751 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:15.052751 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:15.052751 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:15.052751 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:15.052751 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:15.052751 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:15.052751 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:15.052751 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:15.052751 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:15.052751 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:15.053982 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:15.053982 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:15.053982 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:15.053982 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:15.053982 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:49:15.053982 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:49:15.053982 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:49:15.053982 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:49:15.053982 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:49:15.053982 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:49:15.053982 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 14 02:49:15.053982 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:15.054576 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26004d41-d3f5-48ed-9956-63bded601111 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:15.054576 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26004d41-d3f5-48ed-9956-63bded601111 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:15.054576 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26004d41-d3f5-48ed-9956-63bded601111 None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:15.054755 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 150/303] 10.222.0.39 () {56 vars in 1114 bytes} [Fri Oct 14 02:49:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:15.054819 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 154/304] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:49:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:49:15.059243 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c330b3b-c7ac-4779-8293-5b13aa822eef None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:15.063360 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd13cf3-4a9b-4e27-b1d8-34649363b512 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:15.063648 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd13cf3-4a9b-4e27-b1d8-34649363b512 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:15.063924 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd13cf3-4a9b-4e27-b1d8-34649363b512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:15.082786 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-fbd13cf3-4a9b-4e27-b1d8-34649363b512 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:49:15.084505 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-fbd13cf3-4a9b-4e27-b1d8-34649363b512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:49:15.099490 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:15.155332 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 155/305] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:49:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:49:15.156215 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd13cf3-4a9b-4e27-b1d8-34649363b512 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:15.174178 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c330b3b-c7ac-4779-8293-5b13aa822eef None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:15.232692 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbfc7ed-6209-4343-9cc9-c4312acc3b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:15.233228 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbfc7ed-6209-4343-9cc9-c4312acc3b49 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:15.233505 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbfc7ed-6209-4343-9cc9-c4312acc3b49 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:15.233792 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbfc7ed-6209-4343-9cc9-c4312acc3b49 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:15.320385 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:15.325262 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd13cf3-4a9b-4e27-b1d8-34649363b512 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:15.338422 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbfc7ed-6209-4343-9cc9-c4312acc3b49 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:15.340293 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbfc7ed-6209-4343-9cc9-c4312acc3b49 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:15.367897 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a162cff-c846-409c-acc2-08b8c61a7bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:15.368523 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a162cff-c846-409c-acc2-08b8c61a7bec None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:15.368818 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a162cff-c846-409c-acc2-08b8c61a7bec None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:15.369213 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a162cff-c846-409c-acc2-08b8c61a7bec None admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:15.369741 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a162cff-c846-409c-acc2-08b8c61a7bec None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:15.370087 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a162cff-c846-409c-acc2-08b8c61a7bec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:15.371457 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a162cff-c846-409c-acc2-08b8c61a7bec None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:15.389516 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 156/306] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Oct 14 02:49:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:15.411240 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a162cff-c846-409c-acc2-08b8c61a7bec None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:15.433221 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:15.457139 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 151/307] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Oct 14 02:49:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 14 02:49:15.462412 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbfc7ed-6209-4343-9cc9-c4312acc3b49 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:15.529807 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a162cff-c846-409c-acc2-08b8c61a7bec None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:15.550930 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:15.568418 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbfc7ed-6209-4343-9cc9-c4312acc3b49 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:15.581126 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e2e601-5347-4e0b-91aa-a72d3755c3b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:15.581822 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e2e601-5347-4e0b-91aa-a72d3755c3b7 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:15.582062 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e2e601-5347-4e0b-91aa-a72d3755c3b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:15.582350 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e2e601-5347-4e0b-91aa-a72d3755c3b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:15.624199 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1f8a13-a8ac-48a0-b2eb-cbaee1d03e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:15.625161 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1f8a13-a8ac-48a0-b2eb-cbaee1d03e40 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:15.625584 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1f8a13-a8ac-48a0-b2eb-cbaee1d03e40 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:15.626028 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1f8a13-a8ac-48a0-b2eb-cbaee1d03e40 None admin] PATH_INFO: `/v3/projects/0b15574d2808402ca6940d47f10e2db8` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:15.630511 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1f8a13-a8ac-48a0-b2eb-cbaee1d03e40 None admin] RBAC: Authorizing `identity:get_project(project_id=0b15574d2808402ca6940d47f10e2db8)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:15.632141 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1f8a13-a8ac-48a0-b2eb-cbaee1d03e40 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:15.634424 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 152/308] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Oct 14 02:49:15 2022] GET /identity/v3/projects/0b15574d2808402ca6940d47f10e2db8 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:49:15.639339 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1f8a13-a8ac-48a0-b2eb-cbaee1d03e40 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:15.662512 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e2e601-5347-4e0b-91aa-a72d3755c3b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:15.662512 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e2e601-5347-4e0b-91aa-a72d3755c3b7 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:15.678141 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:15.777496 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1f8a13-a8ac-48a0-b2eb-cbaee1d03e40 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:15.815128 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 157/309] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Oct 14 02:49:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 14 02:49:15.817246 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b8dcab-1c45-4b81-b472-b27444cb31bb None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:15.817683 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b8dcab-1c45-4b81-b472-b27444cb31bb None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:15.818607 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b8dcab-1c45-4b81-b472-b27444cb31bb None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:15.820348 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 158/310] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:49:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:49:15.820836 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b066005-8f07-48cf-b95c-eebb30568c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:15.821073 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b8dcab-1c45-4b81-b472-b27444cb31bb None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:15.821494 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b066005-8f07-48cf-b95c-eebb30568c0e None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:15.821838 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b066005-8f07-48cf-b95c-eebb30568c0e None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:15.822123 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b066005-8f07-48cf-b95c-eebb30568c0e None admin] PATH_INFO: `/v3/groups/5903843ac1194f018ce5e748d711cd35` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:15.823311 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b066005-8f07-48cf-b95c-eebb30568c0e None admin] RBAC: Authorizing `identity:get_group(group_id=5903843ac1194f018ce5e748d711cd35)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:15.824783 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b066005-8f07-48cf-b95c-eebb30568c0e None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:15.826717 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 153/311] 10.222.0.39 () {56 vars in 1180 bytes} [Fri Oct 14 02:49:15 2022] GET /identity/v3/groups/5903843ac1194f018ce5e748d711cd35 => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:15.827500 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b066005-8f07-48cf-b95c-eebb30568c0e None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:15.858062 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:15.953911 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b066005-8f07-48cf-b95c-eebb30568c0e None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:15.956361 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b8dcab-1c45-4b81-b472-b27444cb31bb None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:15.970099 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:15.999217 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84a958e-840d-40f4-8be2-79aa50d3a4df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:15.999966 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84a958e-840d-40f4-8be2-79aa50d3a4df None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:16.000305 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84a958e-840d-40f4-8be2-79aa50d3a4df None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:16.000590 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84a958e-840d-40f4-8be2-79aa50d3a4df None admin] PATH_INFO: `/v3/roles/member` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:16.001471 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef408ab4-7407-4583-b5bb-778c47d387a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:16.002146 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef408ab4-7407-4583-b5bb-778c47d387a4 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:16.002429 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef408ab4-7407-4583-b5bb-778c47d387a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:16.002708 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef408ab4-7407-4583-b5bb-778c47d387a4 None admin] PATH_INFO: `/v3/projects/0b15574d2808402ca6940d47f10e2db8` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:16.004537 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef408ab4-7407-4583-b5bb-778c47d387a4 None admin] RBAC: Authorizing `identity:get_project(project_id=0b15574d2808402ca6940d47f10e2db8)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:16.006566 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef408ab4-7407-4583-b5bb-778c47d387a4 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:16.008612 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 154/312] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Oct 14 02:49:15 2022] GET /identity/v3/projects/0b15574d2808402ca6940d47f10e2db8 => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:16.009869 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dac099-4e18-4ae7-9c5a-1abf68c9ef79 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:16.010107 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dac099-4e18-4ae7-9c5a-1abf68c9ef79 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:16.010379 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dac099-4e18-4ae7-9c5a-1abf68c9ef79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:16.030259 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84a958e-840d-40f4-8be2-79aa50d3a4df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:49:16.033329 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-98dac099-4e18-4ae7-9c5a-1abf68c9ef79 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:49:16.034780 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-98dac099-4e18-4ae7-9c5a-1abf68c9ef79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:49:16.035165 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-a84a958e-840d-40f4-8be2-79aa50d3a4df None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 14 02:49:16.035165 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:16.035165 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:16.035165 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:16.035165 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:16.035165 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:16.035165 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:16.035165 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:16.035165 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:16.035165 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:16.035165 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:16.035165 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:16.035165 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:49:16.035165 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:49:16.035165 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:49:16.035165 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:49:16.036024 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:49:16.036024 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:49:16.036024 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:49:16.036024 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:49:16.036024 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:16.036024 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:16.036024 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:16.036024 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:16.036024 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:16.036024 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:16.036024 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:16.036024 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:16.036024 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:16.036024 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:16.036024 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:16.036024 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:16.036848 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:16.036848 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:16.036848 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:16.036848 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:16.036848 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:49:16.036848 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:49:16.036848 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:49:16.036848 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:49:16.036848 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:49:16.036848 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:49:16.036848 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 14 02:49:16.036848 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:49:16.036848 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 159/313] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Oct 14 02:49:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:16.037486 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84a958e-840d-40f4-8be2-79aa50d3a4df None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:16.129392 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 155/314] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:49:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:49:16.132480 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dac099-4e18-4ae7-9c5a-1abf68c9ef79 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:16.160930 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:16.164825 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84a958e-840d-40f4-8be2-79aa50d3a4df None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:16.203949 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5262a9fa-5766-4c84-af9a-4792796ccaf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:16.204556 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5262a9fa-5766-4c84-af9a-4792796ccaf9 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:16.204837 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5262a9fa-5766-4c84-af9a-4792796ccaf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:16.205107 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5262a9fa-5766-4c84-af9a-4792796ccaf9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:16.205867 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5262a9fa-5766-4c84-af9a-4792796ccaf9 None admin] RBAC: Adding query filter params (group.id=5903843ac1194f018ce5e748d711cd35, role.id=9bd0470b544240209a420f640c79c06c, scope.project.id=0b15574d2808402ca6940d47f10e2db8) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:16.206178 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5262a9fa-5766-4c84-af9a-4792796ccaf9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:16.207746 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5262a9fa-5766-4c84-af9a-4792796ccaf9 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:16.214917 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 160/315] 10.222.0.39 () {56 vars in 1376 bytes} [Fri Oct 14 02:49:16 2022] GET /identity/v3/role_assignments?group.id=5903843ac1194f018ce5e748d711cd35&scope.project.id=0b15574d2808402ca6940d47f10e2db8&role.id=9bd0470b544240209a420f640c79c06c => generated 611 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:16.215684 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5262a9fa-5766-4c84-af9a-4792796ccaf9 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:16.257895 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:16.267036 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dac099-4e18-4ae7-9c5a-1abf68c9ef79 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:16.311059 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4598d770-a02a-427c-b26f-a0cffd0be8b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:16.311974 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4598d770-a02a-427c-b26f-a0cffd0be8b3 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:16.312401 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4598d770-a02a-427c-b26f-a0cffd0be8b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:16.312857 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4598d770-a02a-427c-b26f-a0cffd0be8b3 None admin] PATH_INFO: `/v3/groups/5903843ac1194f018ce5e748d711cd35` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:16.314265 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4598d770-a02a-427c-b26f-a0cffd0be8b3 None admin] RBAC: Authorizing `identity:get_group(group_id=5903843ac1194f018ce5e748d711cd35)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:16.316585 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4598d770-a02a-427c-b26f-a0cffd0be8b3 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:16.319603 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 156/316] 10.222.0.39 () {56 vars in 1180 bytes} [Fri Oct 14 02:49:16 2022] GET /identity/v3/groups/5903843ac1194f018ce5e748d711cd35 => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:16.324956 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4598d770-a02a-427c-b26f-a0cffd0be8b3 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:16.338407 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5262a9fa-5766-4c84-af9a-4792796ccaf9 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:16.398115 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab647d5e-20cc-4912-8c68-aa6ea68d7046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:16.398813 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab647d5e-20cc-4912-8c68-aa6ea68d7046 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:16.399157 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab647d5e-20cc-4912-8c68-aa6ea68d7046 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:16.399531 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab647d5e-20cc-4912-8c68-aa6ea68d7046 None admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:16.400588 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab647d5e-20cc-4912-8c68-aa6ea68d7046 None admin] RBAC: Adding query filter params (name=member) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:16.400957 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab647d5e-20cc-4912-8c68-aa6ea68d7046 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:16.403469 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab647d5e-20cc-4912-8c68-aa6ea68d7046 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:16.416771 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 161/317] 10.222.0.39 () {56 vars in 1101 bytes} [Fri Oct 14 02:49:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:16.422755 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab647d5e-20cc-4912-8c68-aa6ea68d7046 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:16.454358 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:16.482054 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4598d770-a02a-427c-b26f-a0cffd0be8b3 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:16.529477 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d549c85e-5dbd-48da-ab55-2ced56df1dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:16.530505 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d549c85e-5dbd-48da-ab55-2ced56df1dc1 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:16.530982 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d549c85e-5dbd-48da-ab55-2ced56df1dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:16.532088 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d549c85e-5dbd-48da-ab55-2ced56df1dc1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:16.533107 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d549c85e-5dbd-48da-ab55-2ced56df1dc1 None admin] RBAC: Adding query filter params (group.id=5903843ac1194f018ce5e748d711cd35, role.id=d695dc72739e4aa7a50d5fb73ad1a91f, scope.project.id=0b15574d2808402ca6940d47f10e2db8) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:16.533648 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d549c85e-5dbd-48da-ab55-2ced56df1dc1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:16.536335 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d549c85e-5dbd-48da-ab55-2ced56df1dc1 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:16.546649 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 157/318] 10.222.0.39 () {56 vars in 1376 bytes} [Fri Oct 14 02:49:16 2022] GET /identity/v3/role_assignments?group.id=5903843ac1194f018ce5e748d711cd35&scope.project.id=0b15574d2808402ca6940d47f10e2db8&role.id=d695dc72739e4aa7a50d5fb73ad1a91f => generated 611 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:49:16.565434 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:16.570864 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab647d5e-20cc-4912-8c68-aa6ea68d7046 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:16.615957 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68c8ac2-868e-431a-be48-16971d6977f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:16.616576 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68c8ac2-868e-431a-be48-16971d6977f8 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:16.616858 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68c8ac2-868e-431a-be48-16971d6977f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:16.617130 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68c8ac2-868e-431a-be48-16971d6977f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:16.637494 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d549c85e-5dbd-48da-ab55-2ced56df1dc1 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:16.714962 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68c8ac2-868e-431a-be48-16971d6977f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:16.717211 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68c8ac2-868e-431a-be48-16971d6977f8 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:16.778850 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d549c85e-5dbd-48da-ab55-2ced56df1dc1 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:16.840550 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd3e04c-6c17-4b98-bd9e-b48abbec2975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:16.840550 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd3e04c-6c17-4b98-bd9e-b48abbec2975 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:16.840550 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd3e04c-6c17-4b98-bd9e-b48abbec2975 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:16.840550 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd3e04c-6c17-4b98-bd9e-b48abbec2975 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:16.845876 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd3e04c-6c17-4b98-bd9e-b48abbec2975 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:49:16.852033 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-3dd3e04c-6c17-4b98-bd9e-b48abbec2975 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 14 02:49:16.852033 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:16.852033 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:16.852033 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:16.852033 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:16.852033 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:16.852033 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:16.852033 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:16.852033 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:16.852033 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:16.852033 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:16.852033 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:16.852033 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:49:16.852033 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:49:16.852033 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:49:16.852033 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:49:16.852963 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:49:16.852963 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:49:16.852963 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:49:16.852963 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:49:16.852963 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:16.852963 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:16.852963 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:16.852963 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:16.852963 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:16.852963 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:16.852963 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:16.852963 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:16.852963 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:16.852963 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:16.852963 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:16.852963 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:16.853851 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:16.853851 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:16.853851 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:16.853851 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:16.853851 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:49:16.853851 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:49:16.853851 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:49:16.853851 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:49:16.853851 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:49:16.853851 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:49:16.853851 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 14 02:49:16.853851 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:16.853851 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 158/319] 10.222.0.39 () {56 vars in 1114 bytes} [Fri Oct 14 02:49:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:16.854852 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0647d9-84ff-4ea5-8463-de604893ca04 None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:16.854936 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0647d9-84ff-4ea5-8463-de604893ca04 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:16.856791 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0647d9-84ff-4ea5-8463-de604893ca04 None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:16.856791 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 159/320] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:49:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:49:16.860472 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:16.861132 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0647d9-84ff-4ea5-8463-de604893ca04 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:16.869894 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 162/321] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Oct 14 02:49:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 14 02:49:16.874030 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de217acd-ebc1-43f8-b1c9-868cb36efde8 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:16.874030 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de217acd-ebc1-43f8-b1c9-868cb36efde8 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:16.874030 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de217acd-ebc1-43f8-b1c9-868cb36efde8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:16.875006 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:16.892287 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-de217acd-ebc1-43f8-b1c9-868cb36efde8 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:49:16.893809 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-de217acd-ebc1-43f8-b1c9-868cb36efde8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:49:16.972771 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 163/322] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:49:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:49:16.972771 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de217acd-ebc1-43f8-b1c9-868cb36efde8 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:16.985429 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0647d9-84ff-4ea5-8463-de604893ca04 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:17.036151 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229380f0-9093-4a40-bcaf-297b6fb280a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:17.036151 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229380f0-9093-4a40-bcaf-297b6fb280a4 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:17.036151 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229380f0-9093-4a40-bcaf-297b6fb280a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:17.036151 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229380f0-9093-4a40-bcaf-297b6fb280a4 None admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:17.036625 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229380f0-9093-4a40-bcaf-297b6fb280a4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:17.036625 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229380f0-9093-4a40-bcaf-297b6fb280a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:17.037732 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229380f0-9093-4a40-bcaf-297b6fb280a4 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:17.048056 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 160/323] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Oct 14 02:49:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:17.054878 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229380f0-9093-4a40-bcaf-297b6fb280a4 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:17.110535 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de217acd-ebc1-43f8-b1c9-868cb36efde8 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:17.142165 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:17.147260 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:17.159617 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e059f2c-d10a-40b2-80fd-55007e1441bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:17.160266 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e059f2c-d10a-40b2-80fd-55007e1441bb None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:17.160553 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e059f2c-d10a-40b2-80fd-55007e1441bb None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:17.160829 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e059f2c-d10a-40b2-80fd-55007e1441bb None admin] PATH_INFO: `/v3/projects/b5c28654bb91452aaa2aff4568067821` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:17.162049 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e059f2c-d10a-40b2-80fd-55007e1441bb None admin] RBAC: Authorizing `identity:get_project(project_id=b5c28654bb91452aaa2aff4568067821)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:17.163516 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e059f2c-d10a-40b2-80fd-55007e1441bb None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:17.165162 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 164/324] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Oct 14 02:49:16 2022] GET /identity/v3/projects/b5c28654bb91452aaa2aff4568067821 => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:17.170583 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e059f2c-d10a-40b2-80fd-55007e1441bb None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:17.182407 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229380f0-9093-4a40-bcaf-297b6fb280a4 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:17.229163 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acaa186-1cea-424e-8d68-f1f4e6a8e5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:17.230497 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acaa186-1cea-424e-8d68-f1f4e6a8e5e4 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:17.230497 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acaa186-1cea-424e-8d68-f1f4e6a8e5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:17.230497 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acaa186-1cea-424e-8d68-f1f4e6a8e5e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:17.294945 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e059f2c-d10a-40b2-80fd-55007e1441bb None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:17.310335 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acaa186-1cea-424e-8d68-f1f4e6a8e5e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:17.312057 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acaa186-1cea-424e-8d68-f1f4e6a8e5e4 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:17.334536 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddc1d1b-e9d1-4c6d-89ee-05715f08a4ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:17.335171 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddc1d1b-e9d1-4c6d-89ee-05715f08a4ee None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:17.335453 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddc1d1b-e9d1-4c6d-89ee-05715f08a4ee None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:17.335725 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddc1d1b-e9d1-4c6d-89ee-05715f08a4ee None admin] PATH_INFO: `/v3/groups/5903843ac1194f018ce5e748d711cd35` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:17.336827 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddc1d1b-e9d1-4c6d-89ee-05715f08a4ee None admin] RBAC: Authorizing `identity:get_group(group_id=5903843ac1194f018ce5e748d711cd35)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:17.338310 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddc1d1b-e9d1-4c6d-89ee-05715f08a4ee None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:17.339905 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 165/325] 10.222.0.39 () {56 vars in 1180 bytes} [Fri Oct 14 02:49:17 2022] GET /identity/v3/groups/5903843ac1194f018ce5e748d711cd35 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:49:17.340677 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddc1d1b-e9d1-4c6d-89ee-05715f08a4ee None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:17.392473 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:17.410054 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:17.435145 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 161/326] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Oct 14 02:49:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 14 02:49:17.436315 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acaa186-1cea-424e-8d68-f1f4e6a8e5e4 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:17.451885 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddc1d1b-e9d1-4c6d-89ee-05715f08a4ee None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:17.498572 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1aa74d4-b739-47a8-b52e-a5205ebb2de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:17.499138 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aa74d4-b739-47a8-b52e-a5205ebb2de4 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:17.499419 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aa74d4-b739-47a8-b52e-a5205ebb2de4 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:17.499692 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aa74d4-b739-47a8-b52e-a5205ebb2de4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:17.518068 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1aa74d4-b739-47a8-b52e-a5205ebb2de4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:49:17.524013 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-a1aa74d4-b739-47a8-b52e-a5205ebb2de4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:49:17.524013 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:17.524013 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:17.524013 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:17.524013 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:17.524013 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:17.524013 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:17.524013 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:17.524013 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:17.524013 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:17.524013 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:17.524013 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:17.524013 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:49:17.524013 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:49:17.524013 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:49:17.524013 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:49:17.524897 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:49:17.524897 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:49:17.524897 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:49:17.524897 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:49:17.524897 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:17.524897 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:17.524897 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:17.524897 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:17.524897 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:17.524897 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:17.524897 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:17.524897 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:17.524897 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:17.524897 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:17.524897 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:17.524897 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:17.525687 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:17.525687 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:17.525687 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:17.525687 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:17.525687 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:49:17.525687 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:49:17.525687 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:49:17.525687 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:49:17.525687 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:49:17.525687 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:49:17.525687 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:49:17.525687 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:49:17.527958 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 166/327] 10.222.0.39 () {56 vars in 1096 bytes} [Fri Oct 14 02:49:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:17.528907 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1aa74d4-b739-47a8-b52e-a5205ebb2de4 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:17.567400 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acaa186-1cea-424e-8d68-f1f4e6a8e5e4 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:17.618922 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09383b0-9826-4e53-9800-3f166a0126b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:17.619858 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09383b0-9826-4e53-9800-3f166a0126b9 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:17.620305 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09383b0-9826-4e53-9800-3f166a0126b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:17.620759 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09383b0-9826-4e53-9800-3f166a0126b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:17.621604 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09383b0-9826-4e53-9800-3f166a0126b9 None admin] RBAC: Adding query filter params (group.id=5903843ac1194f018ce5e748d711cd35, role.id=9bd0470b544240209a420f640c79c06c, scope.project.id=b5c28654bb91452aaa2aff4568067821) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:17.621946 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09383b0-9826-4e53-9800-3f166a0126b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:17.623760 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09383b0-9826-4e53-9800-3f166a0126b9 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:17.632342 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 162/328] 10.222.0.39 () {56 vars in 1376 bytes} [Fri Oct 14 02:49:17 2022] GET /identity/v3/role_assignments?group.id=5903843ac1194f018ce5e748d711cd35&scope.project.id=b5c28654bb91452aaa2aff4568067821&role.id=9bd0470b544240209a420f640c79c06c => generated 611 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:17.634030 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09383b0-9826-4e53-9800-3f166a0126b9 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:17.678983 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1aa74d4-b739-47a8-b52e-a5205ebb2de4 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:17.694262 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:17.706503 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:17.719173 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49584f8-df61-42d7-bf85-4c18987b40e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:17.719820 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49584f8-df61-42d7-bf85-4c18987b40e3 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:17.720101 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49584f8-df61-42d7-bf85-4c18987b40e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:17.720378 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49584f8-df61-42d7-bf85-4c18987b40e3 None admin] PATH_INFO: `/v3/projects/b5c28654bb91452aaa2aff4568067821` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:17.721653 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49584f8-df61-42d7-bf85-4c18987b40e3 None admin] RBAC: Authorizing `identity:get_project(project_id=b5c28654bb91452aaa2aff4568067821)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:17.723188 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49584f8-df61-42d7-bf85-4c18987b40e3 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:17.725150 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 167/329] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Oct 14 02:49:17 2022] GET /identity/v3/projects/b5c28654bb91452aaa2aff4568067821 => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:17.731442 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49584f8-df61-42d7-bf85-4c18987b40e3 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:17.781147 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09383b0-9826-4e53-9800-3f166a0126b9 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:17.840938 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a12559-d1b2-4794-ac1d-aea1e2ab2954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:17.840938 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a12559-d1b2-4794-ac1d-aea1e2ab2954 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:17.841391 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a12559-d1b2-4794-ac1d-aea1e2ab2954 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:17.842617 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a12559-d1b2-4794-ac1d-aea1e2ab2954 None admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:17.842736 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a12559-d1b2-4794-ac1d-aea1e2ab2954 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:17.843123 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a12559-d1b2-4794-ac1d-aea1e2ab2954 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:17.844979 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a12559-d1b2-4794-ac1d-aea1e2ab2954 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:17.856008 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 163/330] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Oct 14 02:49:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:17.861857 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a12559-d1b2-4794-ac1d-aea1e2ab2954 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:17.875328 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49584f8-df61-42d7-bf85-4c18987b40e3 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:17.927968 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fd66b4-974d-4f2a-9603-ddc125ee8f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:17.929409 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fd66b4-974d-4f2a-9603-ddc125ee8f10 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:17.929985 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fd66b4-974d-4f2a-9603-ddc125ee8f10 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:17.930258 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fd66b4-974d-4f2a-9603-ddc125ee8f10 None admin] PATH_INFO: `/v3/groups/5903843ac1194f018ce5e748d711cd35` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:17.931589 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fd66b4-974d-4f2a-9603-ddc125ee8f10 None admin] RBAC: Authorizing `identity:get_group(group_id=5903843ac1194f018ce5e748d711cd35)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:17.933721 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fd66b4-974d-4f2a-9603-ddc125ee8f10 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:17.936651 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 168/331] 10.222.0.39 () {56 vars in 1180 bytes} [Fri Oct 14 02:49:17 2022] GET /identity/v3/groups/5903843ac1194f018ce5e748d711cd35 => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:49:17.941964 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fd66b4-974d-4f2a-9603-ddc125ee8f10 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:17.996163 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:17.999956 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a12559-d1b2-4794-ac1d-aea1e2ab2954 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:18.022426 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:18.039043 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3342b240-ad1e-49ca-a8ad-9808e715db6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:18.040478 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3342b240-ad1e-49ca-a8ad-9808e715db6f None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:18.040478 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3342b240-ad1e-49ca-a8ad-9808e715db6f None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:18.040478 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3342b240-ad1e-49ca-a8ad-9808e715db6f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:18.067726 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fd66b4-974d-4f2a-9603-ddc125ee8f10 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:18.101010 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70bfbf8-69fa-476c-aac4-d0ddcf857c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:18.101610 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70bfbf8-69fa-476c-aac4-d0ddcf857c2b None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:18.101927 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70bfbf8-69fa-476c-aac4-d0ddcf857c2b None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:18.102201 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70bfbf8-69fa-476c-aac4-d0ddcf857c2b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:18.102922 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70bfbf8-69fa-476c-aac4-d0ddcf857c2b None admin] RBAC: Adding query filter params (group.id=5903843ac1194f018ce5e748d711cd35, role.id=d695dc72739e4aa7a50d5fb73ad1a91f, scope.project.id=b5c28654bb91452aaa2aff4568067821) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:18.103232 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70bfbf8-69fa-476c-aac4-d0ddcf857c2b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:18.104942 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70bfbf8-69fa-476c-aac4-d0ddcf857c2b None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:18.108853 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3342b240-ad1e-49ca-a8ad-9808e715db6f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:18.110349 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3342b240-ad1e-49ca-a8ad-9808e715db6f None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:18.110579 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 169/332] 10.222.0.39 () {56 vars in 1376 bytes} [Fri Oct 14 02:49:17 2022] GET /identity/v3/role_assignments?group.id=5903843ac1194f018ce5e748d711cd35&scope.project.id=b5c28654bb91452aaa2aff4568067821&role.id=d695dc72739e4aa7a50d5fb73ad1a91f => generated 611 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:49:18.208958 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:18.213458 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 164/333] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Oct 14 02:49:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 14 02:49:18.219249 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70bfbf8-69fa-476c-aac4-d0ddcf857c2b None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:18.322704 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70bfbf8-69fa-476c-aac4-d0ddcf857c2b None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:18.346210 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:18.357088 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2f885c-9786-44a8-9c76-dab6f2facc78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:18.360635 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2f885c-9786-44a8-9c76-dab6f2facc78 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:18.360635 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2f885c-9786-44a8-9c76-dab6f2facc78 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:18.360635 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2f885c-9786-44a8-9c76-dab6f2facc78 None admin] PATH_INFO: `/v3/projects/e86df8ec1d224299a71e492e2b3fb74d` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:18.360635 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2f885c-9786-44a8-9c76-dab6f2facc78 None admin] RBAC: Authorizing `identity:get_project(project_id=e86df8ec1d224299a71e492e2b3fb74d)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:18.361153 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2f885c-9786-44a8-9c76-dab6f2facc78 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:18.362442 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 170/334] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Oct 14 02:49:18 2022] GET /identity/v3/projects/e86df8ec1d224299a71e492e2b3fb74d => generated 344 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:18.368122 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3342b240-ad1e-49ca-a8ad-9808e715db6f None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:18.471319 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3342b240-ad1e-49ca-a8ad-9808e715db6f None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:18.504685 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b44ce1-5e1f-4540-b5da-f7877e2f94a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:18.505306 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b44ce1-5e1f-4540-b5da-f7877e2f94a8 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:18.506234 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b44ce1-5e1f-4540-b5da-f7877e2f94a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:18.506506 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b44ce1-5e1f-4540-b5da-f7877e2f94a8 None admin] PATH_INFO: `/v3/groups/d0fa86df5fa747f38b9fd14e7fa9e33b` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:18.507608 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b44ce1-5e1f-4540-b5da-f7877e2f94a8 None admin] RBAC: Authorizing `identity:get_group(group_id=d0fa86df5fa747f38b9fd14e7fa9e33b)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:18.509051 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b44ce1-5e1f-4540-b5da-f7877e2f94a8 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:18.510645 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 165/335] 10.222.0.39 () {56 vars in 1180 bytes} [Fri Oct 14 02:49:18 2022] GET /identity/v3/groups/d0fa86df5fa747f38b9fd14e7fa9e33b => generated 229 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:18.516451 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2f885c-9786-44a8-9c76-dab6f2facc78 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:18.612771 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2f885c-9786-44a8-9c76-dab6f2facc78 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:18.645414 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660ee83c-a75e-4198-bed1-781acd3e75c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:18.646045 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ee83c-a75e-4198-bed1-781acd3e75c8 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:18.646324 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ee83c-a75e-4198-bed1-781acd3e75c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:18.646593 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ee83c-a75e-4198-bed1-781acd3e75c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:18.647302 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660ee83c-a75e-4198-bed1-781acd3e75c8 None admin] RBAC: Adding query filter params (group.id=d0fa86df5fa747f38b9fd14e7fa9e33b, role.id=9e86d182c0594e0785b72ba5f390a114, scope.project.id=e86df8ec1d224299a71e492e2b3fb74d) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:18.647606 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660ee83c-a75e-4198-bed1-781acd3e75c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:18.649016 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660ee83c-a75e-4198-bed1-781acd3e75c8 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:18.654568 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 171/336] 10.222.0.39 () {56 vars in 1376 bytes} [Fri Oct 14 02:49:18 2022] GET /identity/v3/role_assignments?group.id=d0fa86df5fa747f38b9fd14e7fa9e33b&scope.project.id=e86df8ec1d224299a71e492e2b3fb74d&role.id=9e86d182c0594e0785b72ba5f390a114 => generated 611 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:21.352533 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003f7716-8d69-4b6f-be0d-3f76a317bf58 None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:21.353415 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003f7716-8d69-4b6f-be0d-3f76a317bf58 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:21.353874 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003f7716-8d69-4b6f-be0d-3f76a317bf58 None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:21.356016 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 166/337] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:49:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:49:21.361023 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7ac048-5127-48ef-a0ef-01c0d4c0e935 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:21.361434 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7ac048-5127-48ef-a0ef-01c0d4c0e935 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:21.361783 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7ac048-5127-48ef-a0ef-01c0d4c0e935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:21.391017 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-4c7ac048-5127-48ef-a0ef-01c0d4c0e935 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:49:21.392231 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-4c7ac048-5127-48ef-a0ef-01c0d4c0e935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:49:21.464063 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67d7837-9f62-43c2-a98a-5f226760b7b2 None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:21.464063 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67d7837-9f62-43c2-a98a-5f226760b7b2 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:21.464063 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67d7837-9f62-43c2-a98a-5f226760b7b2 None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:21.464792 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 167/338] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:49:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 14 02:49:21.469913 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:21.471573 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acce6ccc-f466-4d0e-9644-e36c3c498ae6 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:21.472098 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acce6ccc-f466-4d0e-9644-e36c3c498ae6 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:21.472584 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acce6ccc-f466-4d0e-9644-e36c3c498ae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:21.495778 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-acce6ccc-f466-4d0e-9644-e36c3c498ae6 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:49:21.497325 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-acce6ccc-f466-4d0e-9644-e36c3c498ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:49:21.500026 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 172/339] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:49:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:49:21.571279 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:21.581970 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 168/340] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:49:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 14 02:49:21.707836 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baa40b9-3dd0-4711-9718-c8ebcbc84d35 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:21.709154 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baa40b9-3dd0-4711-9718-c8ebcbc84d35 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:21.709646 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baa40b9-3dd0-4711-9718-c8ebcbc84d35 None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:21.711223 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 173/341] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:49:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:49:21.716643 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c90565-a9c6-4812-9111-d30f0d6f8be1 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:21.716934 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c90565-a9c6-4812-9111-d30f0d6f8be1 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:21.717213 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c90565-a9c6-4812-9111-d30f0d6f8be1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:21.739736 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-64c90565-a9c6-4812-9111-d30f0d6f8be1 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:49:21.740929 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-64c90565-a9c6-4812-9111-d30f0d6f8be1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:49:21.815862 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 169/342] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:49:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:49:21.916626 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baa40b9-3dd0-4711-9718-c8ebcbc84d35 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:21.992141 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06162e88-922f-4b78-888b-eadca61880b6 None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:21.992907 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06162e88-922f-4b78-888b-eadca61880b6 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:21.993406 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06162e88-922f-4b78-888b-eadca61880b6 None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:21.995561 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 170/343] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:49:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 14 02:49:22.002300 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a7c575-f05a-46a6-9d7a-c62e202fa941 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:22.003135 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a7c575-f05a-46a6-9d7a-c62e202fa941 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:22.003638 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a7c575-f05a-46a6-9d7a-c62e202fa941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:22.038769 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-b6a7c575-f05a-46a6-9d7a-c62e202fa941 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:49:22.039970 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-b6a7c575-f05a-46a6-9d7a-c62e202fa941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:49:22.058957 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baa40b9-3dd0-4711-9718-c8ebcbc84d35 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:22.094322 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a74ec32-281b-4f07-91eb-385825a10625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:22.095185 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a74ec32-281b-4f07-91eb-385825a10625 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:22.095605 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a74ec32-281b-4f07-91eb-385825a10625 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:22.096017 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a74ec32-281b-4f07-91eb-385825a10625 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:22.106557 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a74ec32-281b-4f07-91eb-385825a10625 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:22.108186 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a74ec32-281b-4f07-91eb-385825a10625 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:22.111027 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 171/344] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:49:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 14 02:49:22.112027 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a7c575-f05a-46a6-9d7a-c62e202fa941 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:22.123989 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-3a74ec32-281b-4f07-91eb-385825a10625 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:22.123989 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:22.123989 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:22.123989 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:22.123989 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:22.123989 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:22.123989 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:22.123989 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:22.123989 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:22.123989 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:22.123989 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:22.123989 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:22.123989 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:49:22.123989 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:49:22.123989 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:49:22.123989 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:49:22.125039 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:22.125039 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:22.125039 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:22.125039 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:22.125039 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:22.125039 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:22.125039 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:22.125039 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:22.125039 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:22.125039 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:22.125039 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:22.125039 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:22.125039 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:22.125039 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:22.125039 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:22.125039 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:22.125039 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:49:22.126045 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:49:22.126045 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:22.126045 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:49:22.127357 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 174/345] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:49:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:22.133724 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a74ec32-281b-4f07-91eb-385825a10625 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:22.171771 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:22.176074 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:22.216318 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a7c575-f05a-46a6-9d7a-c62e202fa941 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:22.248467 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a74ec32-281b-4f07-91eb-385825a10625 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:22.257944 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c05136-56f2-4364-b9e9-85e07810ff8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:22.258593 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c05136-56f2-4364-b9e9-85e07810ff8b None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:22.258870 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c05136-56f2-4364-b9e9-85e07810ff8b None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:22.259143 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c05136-56f2-4364-b9e9-85e07810ff8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:22.271026 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c05136-56f2-4364-b9e9-85e07810ff8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:22.272879 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c05136-56f2-4364-b9e9-85e07810ff8b None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:22.286683 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c84cef-a07a-4b87-b5a6-5008435068ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:22.287638 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c84cef-a07a-4b87-b5a6-5008435068ab None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:22.288111 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c84cef-a07a-4b87-b5a6-5008435068ab None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:22.288562 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c84cef-a07a-4b87-b5a6-5008435068ab None admin] PATH_INFO: `/v3/domains` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:22.289220 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-51c05136-56f2-4364-b9e9-85e07810ff8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:22.289220 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:22.289220 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:22.289220 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:22.289220 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:22.289220 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:22.289220 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:22.289220 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:22.289220 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:22.289220 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:22.289220 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:22.289220 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:22.289220 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:49:22.289220 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:49:22.289220 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:49:22.289220 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:49:22.290153 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:22.290153 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:22.290153 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:22.290153 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:22.290153 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:22.290153 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:22.290153 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:22.290153 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:22.290153 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:22.290153 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:22.290153 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:22.290153 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:22.290153 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:22.290153 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:22.290153 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:22.290153 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:22.290153 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:49:22.291022 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:49:22.291022 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:22.291022 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:22.291022 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 172/346] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:49:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:22.291232 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c05136-56f2-4364-b9e9-85e07810ff8b None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:22.291834 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c84cef-a07a-4b87-b5a6-5008435068ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:22.292382 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c84cef-a07a-4b87-b5a6-5008435068ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:22.294025 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c84cef-a07a-4b87-b5a6-5008435068ab None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:22.309668 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 175/347] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Oct 14 02:49:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:22.310822 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c84cef-a07a-4b87-b5a6-5008435068ab None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:22.414251 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c05136-56f2-4364-b9e9-85e07810ff8b None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:22.419224 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:22.423665 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c84cef-a07a-4b87-b5a6-5008435068ab None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:22.425103 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:22.457436 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f85eb0-6648-4d66-a97a-9bca6ba09a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:22.458098 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f85eb0-6648-4d66-a97a-9bca6ba09a6c None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:22.458414 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f85eb0-6648-4d66-a97a-9bca6ba09a6c None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:22.458690 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f85eb0-6648-4d66-a97a-9bca6ba09a6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:22.461567 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced66281-aac3-42b4-bcbb-28ca9e996c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:22.462215 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced66281-aac3-42b4-bcbb-28ca9e996c0d None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:22.462494 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced66281-aac3-42b4-bcbb-28ca9e996c0d None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:22.462764 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced66281-aac3-42b4-bcbb-28ca9e996c0d None admin] PATH_INFO: `/v3/domains` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:22.463406 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced66281-aac3-42b4-bcbb-28ca9e996c0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:22.463709 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced66281-aac3-42b4-bcbb-28ca9e996c0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:22.465078 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced66281-aac3-42b4-bcbb-28ca9e996c0d None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:22.469706 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f85eb0-6648-4d66-a97a-9bca6ba09a6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:22.471180 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f85eb0-6648-4d66-a97a-9bca6ba09a6c None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:22.481287 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 176/348] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Oct 14 02:49:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:22.481287 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced66281-aac3-42b4-bcbb-28ca9e996c0d None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:22.490664 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-36f85eb0-6648-4d66-a97a-9bca6ba09a6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:22.490664 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:22.490664 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:22.490664 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:22.490664 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:22.490664 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:22.490664 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:22.490664 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:22.490664 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:22.490664 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:22.490664 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:22.490664 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:22.490664 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:49:22.490664 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:49:22.490664 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:49:22.490664 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:49:22.491573 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:22.491573 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:22.491573 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:22.491573 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:22.491573 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:22.491573 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:22.491573 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:22.491573 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:22.491573 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:22.491573 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:22.491573 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:22.491573 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:22.491573 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:22.491573 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:22.491573 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:22.491573 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:22.491573 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:49:22.492389 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:49:22.492389 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:22.492389 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:22.492389 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 173/349] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:49:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:22.492712 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f85eb0-6648-4d66-a97a-9bca6ba09a6c None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:22.588522 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced66281-aac3-42b4-bcbb-28ca9e996c0d None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:22.595480 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f85eb0-6648-4d66-a97a-9bca6ba09a6c None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:22.623848 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a8c921-d62d-4827-b683-5cdeaaf43c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:22.624440 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a8c921-d62d-4827-b683-5cdeaaf43c2a None admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:22.624734 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a8c921-d62d-4827-b683-5cdeaaf43c2a None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:22.624993 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a8c921-d62d-4827-b683-5cdeaaf43c2a None admin] PATH_INFO: `/v3/users` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:22.625632 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a8c921-d62d-4827-b683-5cdeaaf43c2a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:22.627090 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a8c921-d62d-4827-b683-5cdeaaf43c2a None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:22.630762 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18f7647-f00a-4a17-ae62-463720caf81f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:22.631393 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f7647-f00a-4a17-ae62-463720caf81f None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:22.631670 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f7647-f00a-4a17-ae62-463720caf81f None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:22.631937 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f7647-f00a-4a17-ae62-463720caf81f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:22.642584 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18f7647-f00a-4a17-ae62-463720caf81f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:22.644014 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18f7647-f00a-4a17-ae62-463720caf81f None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:22.658380 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-e18f7647-f00a-4a17-ae62-463720caf81f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:22.658380 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:22.658380 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:22.658380 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:22.658380 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:22.658380 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:22.658380 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:22.658380 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:22.658380 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:22.658380 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:22.658380 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:22.658380 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:22.658380 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:49:22.658380 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:49:22.658380 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:49:22.658380 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:49:22.659366 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:22.659366 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:22.659366 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:22.659366 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:22.659366 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:22.659366 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:22.659366 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:22.659366 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:22.659366 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:22.659366 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:22.659366 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:22.659366 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:22.659366 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:22.659366 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:22.659366 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:22.659366 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:22.659366 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:49:22.660252 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:49:22.660252 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:22.660252 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:22.660252 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 174/350] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:49:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:22.660786 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18f7647-f00a-4a17-ae62-463720caf81f None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:22.676620 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:22.725799 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 177/351] 10.222.0.39 () {60 vars in 1132 bytes} [Fri Oct 14 02:49:22 2022] POST /identity/v3/users => generated 248 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:49:22.725799 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a8c921-d62d-4827-b683-5cdeaaf43c2a None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:22.753611 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:22.793902 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18f7647-f00a-4a17-ae62-463720caf81f None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:22.843530 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc932da-958a-454c-940d-e6a54cd72f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:22.844073 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc932da-958a-454c-940d-e6a54cd72f35 None admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:22.844987 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc932da-958a-454c-940d-e6a54cd72f35 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:22.845273 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc932da-958a-454c-940d-e6a54cd72f35 None admin] PATH_INFO: `/v3/users` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:22.845990 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc932da-958a-454c-940d-e6a54cd72f35 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:22.847844 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc932da-958a-454c-940d-e6a54cd72f35 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:22.860878 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a8c921-d62d-4827-b683-5cdeaaf43c2a None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:22.899582 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2201b66-9e50-4e85-84b4-8c7b4cc6227f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:22.900229 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2201b66-9e50-4e85-84b4-8c7b4cc6227f None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:22.900505 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2201b66-9e50-4e85-84b4-8c7b4cc6227f None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:22.900776 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2201b66-9e50-4e85-84b4-8c7b4cc6227f None admin] PATH_INFO: `/v3/domains` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:22.901430 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2201b66-9e50-4e85-84b4-8c7b4cc6227f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:22.901733 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2201b66-9e50-4e85-84b4-8c7b4cc6227f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:22.903195 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2201b66-9e50-4e85-84b4-8c7b4cc6227f None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:22.916086 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 178/352] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Oct 14 02:49:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:22.917411 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2201b66-9e50-4e85-84b4-8c7b4cc6227f None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:22.927987 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 175/353] 10.222.0.39 () {60 vars in 1132 bytes} [Fri Oct 14 02:49:22 2022] POST /identity/v3/users => generated 246 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:49:22.929565 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff75c6d-9686-4ecb-a4f5-3a9c01e80178 None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:22.930127 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff75c6d-9686-4ecb-a4f5-3a9c01e80178 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:22.930375 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff75c6d-9686-4ecb-a4f5-3a9c01e80178 None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:22.931672 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 176/354] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:49:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:49:22.932406 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff75c6d-9686-4ecb-a4f5-3a9c01e80178 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:23.014313 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:23.028836 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:23.050753 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2201b66-9e50-4e85-84b4-8c7b4cc6227f None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:23.068525 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff75c6d-9686-4ecb-a4f5-3a9c01e80178 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:23.102667 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7926f27-0277-40d3-be03-0e36a4a145fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:23.103697 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7926f27-0277-40d3-be03-0e36a4a145fb None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:23.104189 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7926f27-0277-40d3-be03-0e36a4a145fb None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:23.104939 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7926f27-0277-40d3-be03-0e36a4a145fb None admin] PATH_INFO: `/v3/domains` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:23.106028 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7926f27-0277-40d3-be03-0e36a4a145fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:23.106476 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7926f27-0277-40d3-be03-0e36a4a145fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:23.106857 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef9441d-bbb3-4dcb-a644-6765b22d070a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:23.107575 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef9441d-bbb3-4dcb-a644-6765b22d070a None admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:23.107958 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef9441d-bbb3-4dcb-a644-6765b22d070a None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:23.108328 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef9441d-bbb3-4dcb-a644-6765b22d070a None admin] PATH_INFO: `/v3/users` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:23.108773 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7926f27-0277-40d3-be03-0e36a4a145fb None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:23.109171 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef9441d-bbb3-4dcb-a644-6765b22d070a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:23.111746 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef9441d-bbb3-4dcb-a644-6765b22d070a None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:23.126923 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 179/355] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Oct 14 02:49:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:23.132289 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3305c4c3-5a6e-4401-aba5-e5f090aea069 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:23.132289 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3305c4c3-5a6e-4401-aba5-e5f090aea069 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:23.132289 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3305c4c3-5a6e-4401-aba5-e5f090aea069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:23.148979 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-3305c4c3-5a6e-4401-aba5-e5f090aea069 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:49:23.150338 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-3305c4c3-5a6e-4401-aba5-e5f090aea069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:49:23.226308 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 180/356] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:49:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:49:23.227509 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3305c4c3-5a6e-4401-aba5-e5f090aea069 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:23.274236 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 177/357] 10.222.0.39 () {60 vars in 1132 bytes} [Fri Oct 14 02:49:22 2022] POST /identity/v3/users => generated 248 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:49:23.319342 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:23.369144 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3305c4c3-5a6e-4401-aba5-e5f090aea069 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:23.417158 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e800d3a-140b-4cee-b842-6cc1b09cdd65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:23.417805 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e800d3a-140b-4cee-b842-6cc1b09cdd65 None admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:23.418094 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e800d3a-140b-4cee-b842-6cc1b09cdd65 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:23.418370 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e800d3a-140b-4cee-b842-6cc1b09cdd65 None admin] PATH_INFO: `/v3/users` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:23.419033 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e800d3a-140b-4cee-b842-6cc1b09cdd65 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:23.420505 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e800d3a-140b-4cee-b842-6cc1b09cdd65 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:23.535093 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef9441d-bbb3-4dcb-a644-6765b22d070a None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:23.545035 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 181/358] 10.222.0.39 () {60 vars in 1132 bytes} [Fri Oct 14 02:49:23 2022] POST /identity/v3/users => generated 249 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:49:23.660617 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef9441d-bbb3-4dcb-a644-6765b22d070a None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:23.694614 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:23.712162 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83be9c90-855b-4c6f-b16f-7c4a42e37e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:23.712799 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83be9c90-855b-4c6f-b16f-7c4a42e37e00 None admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:23.713084 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83be9c90-855b-4c6f-b16f-7c4a42e37e00 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:23.713359 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83be9c90-855b-4c6f-b16f-7c4a42e37e00 None admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:23.716051 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83be9c90-855b-4c6f-b16f-7c4a42e37e00 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:23.717502 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83be9c90-855b-4c6f-b16f-7c4a42e37e00 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:23.779255 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.sql.core [None req-83be9c90-855b-4c6f-b16f-7c4a42e37e00 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 14 02:49:23.779255 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 14 02:49:23.779255 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [parameters: {'id': '8c40e5fc55534ab0bdf8d0faa76c8f43', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 14 02:49:23.779255 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81518) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 14 02:49:23.786612 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-83be9c90-855b-4c6f-b16f-7c4a42e37e00 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 14 02:49:23.786612 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:23.786612 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 14 02:49:23.786612 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 14 02:49:23.786612 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 14 02:49:23.786612 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 14 02:49:23.786612 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 14 02:49:23.786612 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application result = self._query(query) Oct 14 02:49:23.786612 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 14 02:49:23.786612 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application conn.query(q) Oct 14 02:49:23.786612 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 14 02:49:23.786612 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 14 02:49:23.786612 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 14 02:49:23.786612 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application result.read() Oct 14 02:49:23.786612 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 14 02:49:23.793340 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 14 02:49:23.793340 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 14 02:49:23.793340 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 14 02:49:23.793340 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 14 02:49:23.793340 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 14 02:49:23.793340 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 14 02:49:23.793340 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 14 02:49:23.793340 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 14 02:49:23.793340 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:23.793340 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 14 02:49:23.793340 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:23.793340 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:23.793340 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 14 02:49:23.793340 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 14 02:49:23.793340 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Oct 14 02:49:23.793340 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return ref.to_dict() Oct 14 02:49:23.793340 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 14 02:49:23.793340 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application next(self.gen) Oct 14 02:49:23.794515 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 14 02:49:23.794515 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application yield resource Oct 14 02:49:23.794515 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 14 02:49:23.794515 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application next(self.gen) Oct 14 02:49:23.794515 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 14 02:49:23.794515 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application self.session.rollback() Oct 14 02:49:23.794515 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 14 02:49:23.794515 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application self.force_reraise() Oct 14 02:49:23.794515 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 14 02:49:23.794515 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise self.value Oct 14 02:49:23.794515 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 14 02:49:23.794515 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 14 02:49:23.794515 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 14 02:49:23.794515 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application session.commit() Oct 14 02:49:23.794515 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 14 02:49:23.794515 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 14 02:49:23.794515 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 14 02:49:23.795361 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application self._prepare_impl() Oct 14 02:49:23.795361 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 14 02:49:23.795361 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application self.session.flush() Oct 14 02:49:23.795361 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 14 02:49:23.795361 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application self._flush(objects) Oct 14 02:49:23.795361 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 14 02:49:23.795361 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 14 02:49:23.795361 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 14 02:49:23.795361 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application compat.raise_( Oct 14 02:49:23.795361 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 14 02:49:23.795361 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception Oct 14 02:49:23.795361 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 14 02:49:23.795361 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application flush_context.execute() Oct 14 02:49:23.795361 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 14 02:49:23.795361 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rec.execute(self) Oct 14 02:49:23.795361 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 14 02:49:23.795361 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 14 02:49:23.796231 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 14 02:49:23.796231 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 14 02:49:23.796231 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 14 02:49:23.796231 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 14 02:49:23.796231 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 14 02:49:23.796231 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 14 02:49:23.796231 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 14 02:49:23.796231 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 14 02:49:23.796231 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 14 02:49:23.796231 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 14 02:49:23.796231 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 14 02:49:23.796231 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 14 02:49:23.796231 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 14 02:49:23.796231 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 14 02:49:23.796231 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 14 02:49:23.796231 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception Oct 14 02:49:23.797060 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 14 02:49:23.797060 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 14 02:49:23.797060 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 14 02:49:23.797060 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 14 02:49:23.797060 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 14 02:49:23.797060 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application result = self._query(query) Oct 14 02:49:23.797060 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 14 02:49:23.797060 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application conn.query(q) Oct 14 02:49:23.797060 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 14 02:49:23.797060 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 14 02:49:23.797060 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 14 02:49:23.797060 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application result.read() Oct 14 02:49:23.797060 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 14 02:49:23.797060 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 14 02:49:23.797060 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 14 02:49:23.797060 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 14 02:49:23.798930 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 14 02:49:23.798930 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 14 02:49:23.798930 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 14 02:49:23.798930 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 14 02:49:23.798930 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 14 02:49:23.798930 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 14 02:49:23.798930 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [parameters: {'id': '8c40e5fc55534ab0bdf8d0faa76c8f43', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 14 02:49:23.798930 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 14 02:49:23.798930 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:23.798930 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 02:49:23.798930 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:23.798930 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:23.798930 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:23.798930 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:23.798930 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:23.798930 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:23.801028 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:23.801028 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:23.801028 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:23.801028 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:23.801028 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:23.801028 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:23.801028 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Oct 14 02:49:23.801028 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Oct 14 02:49:23.801028 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:23.801028 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:23.801028 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Oct 14 02:49:23.801028 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Oct 14 02:49:23.801028 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 14 02:49:23.801028 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 14 02:49:23.801028 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 14 02:49:23.801028 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:23.803899 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 178/359] 10.222.0.39 () {60 vars in 1131 bytes} [Fri Oct 14 02:49:23 2022] POST /identity/v3/roles => generated 144 bytes in 253 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 14 02:49:23.804063 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e800d3a-140b-4cee-b842-6cc1b09cdd65 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:23.956481 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e800d3a-140b-4cee-b842-6cc1b09cdd65 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:23.983426 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:24.004723 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a9e644-1c38-4bce-9574-2d547c2c6bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:24.006398 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a9e644-1c38-4bce-9574-2d547c2c6bbb None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:24.006907 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a9e644-1c38-4bce-9574-2d547c2c6bbb None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:24.007387 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a9e644-1c38-4bce-9574-2d547c2c6bbb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:24.024728 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a9e644-1c38-4bce-9574-2d547c2c6bbb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:49:24.029552 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-29a9e644-1c38-4bce-9574-2d547c2c6bbb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 14 02:49:24.029552 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:24.029552 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:24.029552 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:24.029552 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:24.029552 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:24.029552 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:24.029552 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:24.029552 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:24.029552 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:24.029552 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:24.029552 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:24.029552 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:49:24.029552 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:49:24.029552 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:49:24.029552 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:49:24.036556 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:49:24.036556 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:49:24.036556 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:49:24.036556 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:49:24.036556 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:24.036556 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:24.036556 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:24.036556 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:24.036556 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:24.036556 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:24.036556 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:24.036556 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:24.036556 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:24.036556 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:24.036556 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:24.036556 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:24.037385 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:24.037385 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:24.037385 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:24.037385 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:24.037385 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:49:24.037385 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:49:24.037385 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:49:24.037385 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:49:24.037385 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:49:24.037385 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:49:24.037385 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 14 02:49:24.037385 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:49:24.037385 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 182/360] 10.222.0.39 () {56 vars in 1114 bytes} [Fri Oct 14 02:49:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:24.038123 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83be9c90-855b-4c6f-b16f-7c4a42e37e00 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:24.187545 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83be9c90-855b-4c6f-b16f-7c4a42e37e00 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:24.240518 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999dce2e-2196-4df5-9ce3-28f240cf9a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:24.241482 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999dce2e-2196-4df5-9ce3-28f240cf9a2b None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:24.242001 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999dce2e-2196-4df5-9ce3-28f240cf9a2b None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:24.242485 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999dce2e-2196-4df5-9ce3-28f240cf9a2b None admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:24.243346 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999dce2e-2196-4df5-9ce3-28f240cf9a2b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:24.243872 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999dce2e-2196-4df5-9ce3-28f240cf9a2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:24.245441 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999dce2e-2196-4df5-9ce3-28f240cf9a2b None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:24.257208 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 179/361] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Oct 14 02:49:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:25.524367 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85750adc-97db-4901-a111-650b8866962e None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:25.524891 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85750adc-97db-4901-a111-650b8866962e None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:25.524891 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85750adc-97db-4901-a111-650b8866962e None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:25.526302 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 183/362] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:49:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:49:25.532056 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e744988-d6cc-478d-aadd-b9e0c8a93ca2 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:25.533052 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e744988-d6cc-478d-aadd-b9e0c8a93ca2 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:25.533533 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e744988-d6cc-478d-aadd-b9e0c8a93ca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:25.560117 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-7e744988-d6cc-478d-aadd-b9e0c8a93ca2 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:49:25.561815 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-7e744988-d6cc-478d-aadd-b9e0c8a93ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:49:25.615697 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:25.647584 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 180/363] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:49:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:49:25.673890 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7ff973-30ac-4e6e-bd8c-b1e4aaa0bfb9 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:25.674544 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7ff973-30ac-4e6e-bd8c-b1e4aaa0bfb9 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:25.675037 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7ff973-30ac-4e6e-bd8c-b1e4aaa0bfb9 None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:25.676702 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 184/364] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:49:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:49:25.682973 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d11304-5742-4989-8c2e-807e42442619 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:25.683611 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d11304-5742-4989-8c2e-807e42442619 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:25.684102 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d11304-5742-4989-8c2e-807e42442619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:25.705065 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-72d11304-5742-4989-8c2e-807e42442619 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:49:25.706648 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6446d798-f7d4-40f3-8d96-97819a9d99b7 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:25.706872 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-72d11304-5742-4989-8c2e-807e42442619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:49:25.706985 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6446d798-f7d4-40f3-8d96-97819a9d99b7 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:25.707194 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6446d798-f7d4-40f3-8d96-97819a9d99b7 None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:25.710340 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 185/365] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:49:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:49:25.714573 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0cfa0a-e3fd-43e2-83d2-aa2800321606 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:25.714863 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0cfa0a-e3fd-43e2-83d2-aa2800321606 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:25.715143 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0cfa0a-e3fd-43e2-83d2-aa2800321606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:25.751534 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-1d0cfa0a-e3fd-43e2-83d2-aa2800321606 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:49:25.752742 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-1d0cfa0a-e3fd-43e2-83d2-aa2800321606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:49:25.781262 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 181/366] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:49:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:49:25.866404 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 186/367] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:49:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:49:26.063449 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d11304-5742-4989-8c2e-807e42442619 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:26.172292 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0cfa0a-e3fd-43e2-83d2-aa2800321606 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:26.191105 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d11304-5742-4989-8c2e-807e42442619 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:26.206749 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:26.250529 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d8348d-bef4-4220-a464-78f44252996b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:26.251610 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d8348d-bef4-4220-a464-78f44252996b None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:26.252119 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d8348d-bef4-4220-a464-78f44252996b None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:26.252606 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d8348d-bef4-4220-a464-78f44252996b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:26.267932 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d8348d-bef4-4220-a464-78f44252996b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:49:26.274277 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-61d8348d-bef4-4220-a464-78f44252996b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 14 02:49:26.274277 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:26.274277 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:26.274277 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:26.274277 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:26.274277 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:26.274277 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:26.274277 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:26.274277 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:26.274277 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:26.274277 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:26.274277 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:26.274277 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:49:26.274277 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:49:26.274277 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:49:26.274277 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:49:26.275238 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:49:26.275238 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:49:26.275238 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:49:26.275238 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:49:26.275238 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:26.275238 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:26.275238 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:26.275238 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:26.275238 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:26.275238 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:26.275238 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:26.275238 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:26.275238 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:26.275238 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:26.275238 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:26.275238 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:26.276032 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:26.276032 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:26.276032 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:26.276032 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:26.276032 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:49:26.276032 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:49:26.276032 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:49:26.276032 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:49:26.276032 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:49:26.276032 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:49:26.276032 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 14 02:49:26.276032 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:26.276032 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 182/368] 10.222.0.39 () {56 vars in 1102 bytes} [Fri Oct 14 02:49:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:26.276714 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d8348d-bef4-4220-a464-78f44252996b None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:26.334057 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0cfa0a-e3fd-43e2-83d2-aa2800321606 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:26.350543 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:26.382408 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdc5959-1280-43f5-af3d-b8303cbfccdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:26.383515 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc5959-1280-43f5-af3d-b8303cbfccdd None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:26.383833 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc5959-1280-43f5-af3d-b8303cbfccdd None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:26.384438 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc5959-1280-43f5-af3d-b8303cbfccdd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:26.394935 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdc5959-1280-43f5-af3d-b8303cbfccdd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:49:26.397058 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d8348d-bef4-4220-a464-78f44252996b None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:26.399873 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-efdc5959-1280-43f5-af3d-b8303cbfccdd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 14 02:49:26.399873 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:26.399873 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:26.399873 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:26.399873 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:26.399873 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:26.399873 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:26.399873 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:26.399873 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:26.399873 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:26.399873 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:26.399873 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:26.399873 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:49:26.399873 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:49:26.399873 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:49:26.399873 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:49:26.400761 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:49:26.400761 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:49:26.400761 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:49:26.400761 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:49:26.400761 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:26.400761 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:26.400761 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:26.400761 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:26.400761 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:26.400761 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:26.400761 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:26.400761 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:26.400761 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:26.400761 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:26.400761 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:26.400761 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:26.401557 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:26.401557 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:26.401557 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:26.401557 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:26.401557 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:49:26.401557 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:49:26.401557 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:49:26.401557 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:49:26.401557 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:49:26.401557 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:49:26.401557 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 14 02:49:26.401557 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:49:26.401557 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 187/369] 10.222.0.39 () {56 vars in 1102 bytes} [Fri Oct 14 02:49:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:26.402260 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdc5959-1280-43f5-af3d-b8303cbfccdd None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:26.440171 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e3aa5b-5862-43a7-924a-ab496c1f1b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:26.440804 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e3aa5b-5862-43a7-924a-ab496c1f1b54 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:26.441085 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e3aa5b-5862-43a7-924a-ab496c1f1b54 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:26.441360 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e3aa5b-5862-43a7-924a-ab496c1f1b54 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:26.451546 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e3aa5b-5862-43a7-924a-ab496c1f1b54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:49:26.457093 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-e9e3aa5b-5862-43a7-924a-ab496c1f1b54 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 14 02:49:26.457093 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:26.457093 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:26.457093 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:26.457093 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:26.457093 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:26.457093 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:26.457093 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:26.457093 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:26.457093 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:26.457093 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:26.457093 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:26.457093 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:49:26.457093 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:49:26.457093 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:49:26.457093 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:49:26.458037 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:49:26.458037 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:49:26.458037 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:49:26.458037 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:49:26.458037 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:26.458037 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:26.458037 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:26.458037 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:26.458037 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:26.458037 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:26.458037 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:26.458037 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:26.458037 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:26.458037 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:26.458037 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:26.458037 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:26.458866 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:26.458866 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:26.458866 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:26.458866 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:26.458866 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:49:26.458866 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:49:26.458866 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:49:26.458866 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:49:26.458866 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:49:26.458866 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:49:26.458866 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 14 02:49:26.458866 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:26.458866 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 183/370] 10.222.0.39 () {56 vars in 1102 bytes} [Fri Oct 14 02:49:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:26.461586 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae58120-9281-47a1-a69a-598c7b361437 None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:26.462056 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae58120-9281-47a1-a69a-598c7b361437 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:26.462335 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae58120-9281-47a1-a69a-598c7b361437 None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:26.463610 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 184/371] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:49:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:49:26.464313 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae58120-9281-47a1-a69a-598c7b361437 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:26.496985 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:26.521076 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdc5959-1280-43f5-af3d-b8303cbfccdd None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:26.558934 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4c0257-2444-4996-9c9b-a511beff202b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:26.559720 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4c0257-2444-4996-9c9b-a511beff202b None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:26.560000 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4c0257-2444-4996-9c9b-a511beff202b None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:26.560214 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4c0257-2444-4996-9c9b-a511beff202b None admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:26.560870 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4c0257-2444-4996-9c9b-a511beff202b None admin] RBAC: Adding query filter params (name=service) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:26.561175 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4c0257-2444-4996-9c9b-a511beff202b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:26.562613 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4c0257-2444-4996-9c9b-a511beff202b None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:26.571010 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 188/372] 10.222.0.39 () {56 vars in 1103 bytes} [Fri Oct 14 02:49:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:26.575490 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4c0257-2444-4996-9c9b-a511beff202b None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:26.576325 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae58120-9281-47a1-a69a-598c7b361437 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:26.611615 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:26.613598 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094d3f3b-4feb-441a-8292-348fa3aafd35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:26.614237 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094d3f3b-4feb-441a-8292-348fa3aafd35 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:26.615123 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094d3f3b-4feb-441a-8292-348fa3aafd35 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:26.615402 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094d3f3b-4feb-441a-8292-348fa3aafd35 None admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:26.616126 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094d3f3b-4feb-441a-8292-348fa3aafd35 None admin] RBAC: Adding query filter params (name=service) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:26.616442 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094d3f3b-4feb-441a-8292-348fa3aafd35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:26.617862 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094d3f3b-4feb-441a-8292-348fa3aafd35 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:26.626453 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 185/373] 10.222.0.39 () {56 vars in 1103 bytes} [Fri Oct 14 02:49:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:26.627651 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdff408-6a5b-4777-8fe7-49fb9eacb71c None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:26.627934 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdff408-6a5b-4777-8fe7-49fb9eacb71c None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:26.628209 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdff408-6a5b-4777-8fe7-49fb9eacb71c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:26.647543 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-3cdff408-6a5b-4777-8fe7-49fb9eacb71c None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:49:26.648857 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-3cdff408-6a5b-4777-8fe7-49fb9eacb71c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:49:26.685239 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4c0257-2444-4996-9c9b-a511beff202b None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:26.718570 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 186/374] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:49:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:49:26.719687 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdff408-6a5b-4777-8fe7-49fb9eacb71c None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:26.728263 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10933646-8b97-4ca0-b0d4-61caa21c5de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:26.728881 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10933646-8b97-4ca0-b0d4-61caa21c5de3 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:26.729168 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10933646-8b97-4ca0-b0d4-61caa21c5de3 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:26.729700 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10933646-8b97-4ca0-b0d4-61caa21c5de3 None admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:26.730405 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10933646-8b97-4ca0-b0d4-61caa21c5de3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:26.730713 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10933646-8b97-4ca0-b0d4-61caa21c5de3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:26.732092 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10933646-8b97-4ca0-b0d4-61caa21c5de3 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:26.742396 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 189/375] 10.222.0.39 () {56 vars in 1103 bytes} [Fri Oct 14 02:49:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:26.743217 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10933646-8b97-4ca0-b0d4-61caa21c5de3 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:26.751889 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:26.847324 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:26.851463 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdff408-6a5b-4777-8fe7-49fb9eacb71c None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:26.855139 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10933646-8b97-4ca0-b0d4-61caa21c5de3 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:26.896961 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0b0f77-6d31-4257-8759-4123cb3c24fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:26.897311 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b388f93-3a59-4197-9b2d-9c55fc91e954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:26.897515 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b0f77-6d31-4257-8759-4123cb3c24fe None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:26.897836 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b0f77-6d31-4257-8759-4123cb3c24fe None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:26.898118 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b0f77-6d31-4257-8759-4123cb3c24fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:26.898808 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b388f93-3a59-4197-9b2d-9c55fc91e954 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:26.899267 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b388f93-3a59-4197-9b2d-9c55fc91e954 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:26.899726 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b388f93-3a59-4197-9b2d-9c55fc91e954 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:26.910504 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0b0f77-6d31-4257-8759-4123cb3c24fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:26.912163 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b388f93-3a59-4197-9b2d-9c55fc91e954 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:26.912335 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0b0f77-6d31-4257-8759-4123cb3c24fe None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:26.914049 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b388f93-3a59-4197-9b2d-9c55fc91e954 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:26.931570 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-6e0b0f77-6d31-4257-8759-4123cb3c24fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:26.931570 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:26.931570 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:26.931570 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:26.931570 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:26.931570 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:26.931570 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:26.931570 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:26.931570 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:26.931570 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:26.931570 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:26.931570 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:26.931570 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:49:26.931570 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:49:26.931570 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:49:26.931570 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:49:26.932552 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:26.932552 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:26.932552 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:26.932552 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:26.932552 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:26.932552 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:26.932552 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:26.932552 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:26.932552 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:26.932552 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:26.932552 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:26.932552 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:26.932552 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:26.932552 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:26.932552 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:26.932552 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:26.932552 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:49:26.933419 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:49:26.933419 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:26.933419 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:49:26.933419 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 190/376] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:49:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:26.933642 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0b0f77-6d31-4257-8759-4123cb3c24fe None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:26.941073 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-7b388f93-3a59-4197-9b2d-9c55fc91e954 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:26.941073 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:26.941073 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:26.941073 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:26.941073 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:26.941073 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:26.941073 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:26.941073 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:26.941073 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:26.941073 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:26.941073 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:26.941073 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:26.941073 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:49:26.941073 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:49:26.941073 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:49:26.941073 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:49:26.942248 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:26.942248 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:26.942248 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:26.942248 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:26.942248 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:26.942248 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:26.942248 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:26.942248 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:26.942248 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:26.942248 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:26.942248 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:26.942248 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:26.942248 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:26.942248 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:26.942248 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:26.942248 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:26.942248 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:49:26.943684 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:49:26.943684 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:26.943684 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:26.945272 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 187/377] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:49:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:26.946222 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b388f93-3a59-4197-9b2d-9c55fc91e954 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:27.040895 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:27.056145 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0b0f77-6d31-4257-8759-4123cb3c24fe None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:27.066616 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b388f93-3a59-4197-9b2d-9c55fc91e954 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:27.089856 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125f75f3-4e8c-4353-87a2-317fbb9ae607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:27.090482 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125f75f3-4e8c-4353-87a2-317fbb9ae607 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:27.090764 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125f75f3-4e8c-4353-87a2-317fbb9ae607 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:27.091035 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125f75f3-4e8c-4353-87a2-317fbb9ae607 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:27.099623 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9c14a5-75bb-4aaa-8c86-47afdb20584f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:27.100224 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9c14a5-75bb-4aaa-8c86-47afdb20584f None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:27.100501 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9c14a5-75bb-4aaa-8c86-47afdb20584f None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:27.100779 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9c14a5-75bb-4aaa-8c86-47afdb20584f None admin] PATH_INFO: `/v3/domains` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:27.101325 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125f75f3-4e8c-4353-87a2-317fbb9ae607 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:27.101425 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9c14a5-75bb-4aaa-8c86-47afdb20584f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:27.101712 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9c14a5-75bb-4aaa-8c86-47afdb20584f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:27.102809 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125f75f3-4e8c-4353-87a2-317fbb9ae607 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:27.103091 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9c14a5-75bb-4aaa-8c86-47afdb20584f None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:27.109772 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:27.115345 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 188/378] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Oct 14 02:49:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:27.122757 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9c14a5-75bb-4aaa-8c86-47afdb20584f None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:27.122872 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-125f75f3-4e8c-4353-87a2-317fbb9ae607 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:27.122872 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:27.122872 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:27.122872 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:27.122872 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:27.122872 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:27.122872 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:27.122872 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:27.122872 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:27.122872 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:27.122872 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:27.122872 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:27.122872 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:49:27.122872 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:49:27.122872 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:49:27.122872 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:49:27.123775 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:27.123775 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:27.123775 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:27.123775 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:27.123775 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:27.123775 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:27.123775 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:27.123775 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:27.123775 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:27.123775 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:27.123775 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:27.123775 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:27.123775 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:27.123775 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:27.123775 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:27.123775 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:27.123775 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:49:27.124592 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:49:27.124592 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:27.124592 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:49:27.124592 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 191/379] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:49:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:27.124592 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125f75f3-4e8c-4353-87a2-317fbb9ae607 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:27.215518 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9c14a5-75bb-4aaa-8c86-47afdb20584f None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:27.219762 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125f75f3-4e8c-4353-87a2-317fbb9ae607 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:27.248925 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccbe69c-7e2b-4e60-b01d-cf9479dd6a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:27.249523 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccbe69c-7e2b-4e60-b01d-cf9479dd6a6d None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:27.249838 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccbe69c-7e2b-4e60-b01d-cf9479dd6a6d None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:27.250115 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccbe69c-7e2b-4e60-b01d-cf9479dd6a6d None admin] PATH_INFO: `/v3/domains` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:27.250754 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccbe69c-7e2b-4e60-b01d-cf9479dd6a6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:27.251061 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccbe69c-7e2b-4e60-b01d-cf9479dd6a6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:27.252439 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccbe69c-7e2b-4e60-b01d-cf9479dd6a6d None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:27.253475 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dd7bd2-5bc8-4e4e-aded-7f566eeba206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:27.253928 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:27.254128 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd7bd2-5bc8-4e4e-aded-7f566eeba206 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:27.254413 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd7bd2-5bc8-4e4e-aded-7f566eeba206 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:27.254690 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd7bd2-5bc8-4e4e-aded-7f566eeba206 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:27.263888 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 189/380] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Oct 14 02:49:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:27.265040 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce9d70a-2135-4c43-b428-ea06cb4fb8f0 None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:27.265346 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce9d70a-2135-4c43-b428-ea06cb4fb8f0 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:27.265619 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce9d70a-2135-4c43-b428-ea06cb4fb8f0 None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:27.266899 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 190/381] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:49:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:49:27.267602 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce9d70a-2135-4c43-b428-ea06cb4fb8f0 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:27.269659 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dd7bd2-5bc8-4e4e-aded-7f566eeba206 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:49:27.275155 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-96dd7bd2-5bc8-4e4e-aded-7f566eeba206 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 14 02:49:27.275155 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:27.275155 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:27.275155 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:27.275155 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:27.275155 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:27.275155 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:27.275155 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:27.275155 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:27.275155 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:27.275155 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:27.275155 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:27.275155 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:49:27.275155 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:49:27.275155 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:49:27.275155 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:49:27.276141 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:49:27.276141 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:49:27.276141 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:49:27.276141 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:49:27.276141 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:27.276141 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:27.276141 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:27.276141 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:27.276141 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:27.276141 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:27.276141 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:27.276141 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:27.276141 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:27.276141 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:27.276141 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:27.276141 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:27.276930 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:27.276930 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:27.276930 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:27.276930 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:27.276930 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:49:27.276930 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:49:27.276930 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:49:27.276930 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:49:27.276930 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:49:27.276930 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:49:27.276930 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 14 02:49:27.276930 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:49:27.276930 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 192/382] 10.222.0.39 () {56 vars in 1102 bytes} [Fri Oct 14 02:49:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:27.277744 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dd7bd2-5bc8-4e4e-aded-7f566eeba206 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:27.334639 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:27.369584 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce9d70a-2135-4c43-b428-ea06cb4fb8f0 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:27.376133 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dd7bd2-5bc8-4e4e-aded-7f566eeba206 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:27.403158 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2345c6-8d89-406e-b72f-4eb7b086cf2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:27.404093 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2345c6-8d89-406e-b72f-4eb7b086cf2e None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:27.404384 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2345c6-8d89-406e-b72f-4eb7b086cf2e None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:27.404661 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2345c6-8d89-406e-b72f-4eb7b086cf2e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:27.408154 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c83c18-1703-47ac-b6b8-301c95ccb1cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:27.408751 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c83c18-1703-47ac-b6b8-301c95ccb1cf None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:27.409034 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c83c18-1703-47ac-b6b8-301c95ccb1cf None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:27.409308 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c83c18-1703-47ac-b6b8-301c95ccb1cf None admin] PATH_INFO: `/v3/domains` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:27.409978 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c83c18-1703-47ac-b6b8-301c95ccb1cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:27.410283 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c83c18-1703-47ac-b6b8-301c95ccb1cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:27.412218 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c83c18-1703-47ac-b6b8-301c95ccb1cf None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:27.412424 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2345c6-8d89-406e-b72f-4eb7b086cf2e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:27.413893 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2345c6-8d89-406e-b72f-4eb7b086cf2e None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:27.423664 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 193/383] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Oct 14 02:49:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:27.424283 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c83c18-1703-47ac-b6b8-301c95ccb1cf None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:27.426166 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-3d2345c6-8d89-406e-b72f-4eb7b086cf2e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 14 02:49:27.426166 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:27.426166 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:27.426166 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:27.426166 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:27.426166 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:27.426166 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:27.426166 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:27.426166 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:27.426166 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:27.426166 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:27.426166 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:27.426166 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 14 02:49:27.426166 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 14 02:49:27.426166 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 14 02:49:27.426166 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 14 02:49:27.427629 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:27.427629 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:27.427629 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 14 02:49:27.427629 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:49:27.427629 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 14 02:49:27.427629 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:49:27.427629 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:27.427629 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:27.427629 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:27.427629 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:27.427629 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:27.427629 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:27.427629 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:27.427629 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:27.427629 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:27.427629 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:27.427629 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:27.427629 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:27.429259 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:27.429259 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:27.429259 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 14 02:49:27.429259 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 14 02:49:27.429259 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 14 02:49:27.429259 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 14 02:49:27.429259 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 14 02:49:27.429259 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 14 02:49:27.429259 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 14 02:49:27.429259 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:27.429259 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 191/384] 10.222.0.39 () {56 vars in 1093 bytes} [Fri Oct 14 02:49:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:27.429259 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3531aa-f1ea-4da4-9c20-f02cd63142a7 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:27.429936 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3531aa-f1ea-4da4-9c20-f02cd63142a7 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:27.429936 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3531aa-f1ea-4da4-9c20-f02cd63142a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:27.446977 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-9e3531aa-f1ea-4da4-9c20-f02cd63142a7 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:49:27.448181 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-9e3531aa-f1ea-4da4-9c20-f02cd63142a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:49:27.496607 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:27.513244 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 192/385] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:49:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:49:27.514079 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3531aa-f1ea-4da4-9c20-f02cd63142a7 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:27.524465 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c83c18-1703-47ac-b6b8-301c95ccb1cf None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:27.543269 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:27.560384 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafd173f-4023-4bba-a29f-bb4d62dd579e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:27.562761 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd173f-4023-4bba-a29f-bb4d62dd579e None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:27.562761 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd173f-4023-4bba-a29f-bb4d62dd579e None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:27.562761 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd173f-4023-4bba-a29f-bb4d62dd579e None admin] PATH_INFO: `/v3/users/glance` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:27.569402 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafd173f-4023-4bba-a29f-bb4d62dd579e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:27.571271 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafd173f-4023-4bba-a29f-bb4d62dd579e None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:27.585241 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-eafd173f-4023-4bba-a29f-bb4d62dd579e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 14 02:49:27.585241 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:27.585241 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:27.585241 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:27.585241 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:27.585241 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:27.585241 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:27.585241 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:27.585241 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:27.585241 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:27.585241 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:27.585241 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:27.585241 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 14 02:49:27.585241 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 14 02:49:27.585241 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 14 02:49:27.585241 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 14 02:49:27.591164 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:27.591164 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:27.591164 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 14 02:49:27.591164 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:49:27.591164 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 14 02:49:27.591164 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:49:27.591164 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:27.591164 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:27.591164 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:27.591164 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:27.591164 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:27.591164 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:27.591164 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:27.591164 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:27.591164 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:27.591164 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:27.591164 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:27.591164 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:27.592076 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:27.592076 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:27.592076 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 14 02:49:27.592076 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 14 02:49:27.592076 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 14 02:49:27.592076 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 14 02:49:27.592076 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 14 02:49:27.592076 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 14 02:49:27.592076 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 14 02:49:27.592076 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:49:27.592076 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 194/386] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Oct 14 02:49:27 2022] GET /identity/v3/users/glance => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:27.592076 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafd173f-4023-4bba-a29f-bb4d62dd579e None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:27.638703 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3531aa-f1ea-4da4-9c20-f02cd63142a7 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:27.677025 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4846eddc-40fa-477a-aff5-c7982db35eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:27.677632 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4846eddc-40fa-477a-aff5-c7982db35eed None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:27.677953 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4846eddc-40fa-477a-aff5-c7982db35eed None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:27.678232 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4846eddc-40fa-477a-aff5-c7982db35eed None admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:27.678890 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4846eddc-40fa-477a-aff5-c7982db35eed None admin] RBAC: Adding query filter params (name=service) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:27.679321 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4846eddc-40fa-477a-aff5-c7982db35eed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:27.680767 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4846eddc-40fa-477a-aff5-c7982db35eed None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:27.688668 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 193/387] 10.222.0.39 () {56 vars in 1103 bytes} [Fri Oct 14 02:49:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:27.689442 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4846eddc-40fa-477a-aff5-c7982db35eed None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:27.703532 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafd173f-4023-4bba-a29f-bb4d62dd579e None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:27.737995 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:27.741243 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1f2af0-6bbf-4172-91be-519eaf9e1a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:27.741911 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1f2af0-6bbf-4172-91be-519eaf9e1a66 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:27.742199 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1f2af0-6bbf-4172-91be-519eaf9e1a66 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:27.742476 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1f2af0-6bbf-4172-91be-519eaf9e1a66 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:27.751512 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1f2af0-6bbf-4172-91be-519eaf9e1a66 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:27.752233 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1f2af0-6bbf-4172-91be-519eaf9e1a66 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:27.763904 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-ae1f2af0-6bbf-4172-91be-519eaf9e1a66 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 14 02:49:27.763904 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:27.763904 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:27.763904 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:27.763904 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:27.763904 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:27.763904 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:27.763904 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:27.763904 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:27.763904 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:27.763904 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:27.763904 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:27.763904 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 14 02:49:27.763904 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 14 02:49:27.763904 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 14 02:49:27.763904 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 14 02:49:27.764803 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:27.764803 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:27.764803 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 14 02:49:27.764803 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:49:27.764803 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 14 02:49:27.764803 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:49:27.764803 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:27.764803 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:27.764803 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:27.764803 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:27.764803 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:27.764803 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:27.764803 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:27.764803 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:27.764803 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:27.764803 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:27.764803 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:27.764803 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:27.765683 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:27.765683 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:27.765683 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 14 02:49:27.765683 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 14 02:49:27.765683 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 14 02:49:27.765683 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 14 02:49:27.765683 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 14 02:49:27.765683 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 14 02:49:27.765683 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 14 02:49:27.765683 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:49:27.769822 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 195/388] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Oct 14 02:49:27 2022] GET /identity/v3/users/cinder => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:27.771226 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1f2af0-6bbf-4172-91be-519eaf9e1a66 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:27.794206 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4846eddc-40fa-477a-aff5-c7982db35eed None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:27.810283 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:27.826277 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3cd111-ce2e-4a62-8b4d-27c282ab3f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:27.826886 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3cd111-ce2e-4a62-8b4d-27c282ab3f1a None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:27.827167 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3cd111-ce2e-4a62-8b4d-27c282ab3f1a None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:27.827437 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3cd111-ce2e-4a62-8b4d-27c282ab3f1a None admin] PATH_INFO: `/v3/users` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:27.828163 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3cd111-ce2e-4a62-8b4d-27c282ab3f1a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:27.828453 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3cd111-ce2e-4a62-8b4d-27c282ab3f1a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:27.829977 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3cd111-ce2e-4a62-8b4d-27c282ab3f1a None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:27.866533 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 194/389] 10.222.0.39 () {56 vars in 1133 bytes} [Fri Oct 14 02:49:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:27.867624 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3cd111-ce2e-4a62-8b4d-27c282ab3f1a None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:27.874051 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1f2af0-6bbf-4172-91be-519eaf9e1a66 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:27.908510 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3d478f-8628-4b4f-bdd1-1c94658eb614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:27.909115 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3d478f-8628-4b4f-bdd1-1c94658eb614 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:27.909397 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3d478f-8628-4b4f-bdd1-1c94658eb614 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:27.909675 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3d478f-8628-4b4f-bdd1-1c94658eb614 None admin] PATH_INFO: `/v3/users` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:27.910430 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3d478f-8628-4b4f-bdd1-1c94658eb614 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:27.910738 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3d478f-8628-4b4f-bdd1-1c94658eb614 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:27.912195 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3d478f-8628-4b4f-bdd1-1c94658eb614 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:27.947618 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 196/390] 10.222.0.39 () {56 vars in 1137 bytes} [Fri Oct 14 02:49:27 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:27.949429 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3d478f-8628-4b4f-bdd1-1c94658eb614 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:27.966149 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3cd111-ce2e-4a62-8b4d-27c282ab3f1a None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:27.968397 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:28.000806 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8a9833-2430-4861-bf12-44778c35af56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:28.001433 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8a9833-2430-4861-bf12-44778c35af56 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:28.001900 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8a9833-2430-4861-bf12-44778c35af56 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:28.002069 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8a9833-2430-4861-bf12-44778c35af56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:28.013708 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8a9833-2430-4861-bf12-44778c35af56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:28.015201 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8a9833-2430-4861-bf12-44778c35af56 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:28.030093 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-1c8a9833-2430-4861-bf12-44778c35af56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:28.030093 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:28.030093 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:28.030093 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:28.030093 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:28.030093 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:28.030093 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:28.030093 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:28.030093 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:28.030093 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:28.030093 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:28.030093 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:28.030093 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:49:28.030093 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:49:28.030093 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:49:28.030093 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:49:28.030979 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:28.030979 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:28.030979 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:28.030979 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:28.030979 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:28.030979 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:28.030979 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:28.030979 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:28.030979 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:28.030979 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:28.030979 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:28.030979 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:28.030979 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:28.030979 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:28.030979 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:28.030979 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:28.030979 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:49:28.031822 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:49:28.031822 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:28.031822 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:28.031822 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 195/391] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:49:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:28.032042 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8a9833-2430-4861-bf12-44778c35af56 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:28.045910 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:28.051628 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3d478f-8628-4b4f-bdd1-1c94658eb614 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:28.086152 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8728e63e-bbbe-4b26-bc29-b45dc5e1a2f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:28.086760 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8728e63e-bbbe-4b26-bc29-b45dc5e1a2f7 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:28.087036 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8728e63e-bbbe-4b26-bc29-b45dc5e1a2f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:28.087308 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8728e63e-bbbe-4b26-bc29-b45dc5e1a2f7 None admin] PATH_INFO: `/v3/users` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:28.088024 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8728e63e-bbbe-4b26-bc29-b45dc5e1a2f7 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:28.088332 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8728e63e-bbbe-4b26-bc29-b45dc5e1a2f7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:28.089787 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8728e63e-bbbe-4b26-bc29-b45dc5e1a2f7 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:28.106614 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 197/392] 10.222.0.39 () {56 vars in 1137 bytes} [Fri Oct 14 02:49:27 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:28.107399 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8728e63e-bbbe-4b26-bc29-b45dc5e1a2f7 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:28.132077 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8a9833-2430-4861-bf12-44778c35af56 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:28.165119 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5951aeaf-9108-4d27-860e-53f31fe2146c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:28.165792 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5951aeaf-9108-4d27-860e-53f31fe2146c None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:28.166090 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5951aeaf-9108-4d27-860e-53f31fe2146c None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:28.166352 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5951aeaf-9108-4d27-860e-53f31fe2146c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:28.176589 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5951aeaf-9108-4d27-860e-53f31fe2146c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:28.178093 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5951aeaf-9108-4d27-860e-53f31fe2146c None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:28.192105 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-5951aeaf-9108-4d27-860e-53f31fe2146c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:28.192105 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:28.192105 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:28.192105 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:28.192105 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:28.192105 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:28.192105 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:28.192105 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:28.192105 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:28.192105 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:28.192105 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:28.192105 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:28.192105 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:49:28.192105 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:49:28.192105 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:49:28.192105 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:49:28.193563 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:28.193563 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:28.193563 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:28.193563 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:28.193563 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:28.193563 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:28.193563 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:28.193563 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:28.193563 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:28.193563 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:28.193563 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:28.193563 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:28.193563 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:28.193563 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:28.193563 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:28.193563 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:28.193563 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:49:28.194436 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:49:28.194436 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:28.194436 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:28.194436 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 196/393] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:49:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:28.194649 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5951aeaf-9108-4d27-860e-53f31fe2146c None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:28.205792 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:28.206861 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8728e63e-bbbe-4b26-bc29-b45dc5e1a2f7 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:28.240766 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56895616-b809-4bc3-ba8a-294dc8ef36ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:28.241396 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56895616-b809-4bc3-ba8a-294dc8ef36ab None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:28.241681 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56895616-b809-4bc3-ba8a-294dc8ef36ab None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:28.241986 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56895616-b809-4bc3-ba8a-294dc8ef36ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:28.251612 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56895616-b809-4bc3-ba8a-294dc8ef36ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:28.253125 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56895616-b809-4bc3-ba8a-294dc8ef36ab None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:28.269044 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-56895616-b809-4bc3-ba8a-294dc8ef36ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:28.269044 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:28.269044 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:28.269044 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:28.269044 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:28.269044 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:28.269044 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:28.269044 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:28.269044 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:28.269044 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:28.269044 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:28.269044 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:28.269044 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:49:28.269044 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:49:28.269044 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:49:28.269044 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:49:28.270105 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:28.270105 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:28.270105 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:28.270105 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:28.270105 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:28.270105 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:28.270105 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:28.270105 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:28.270105 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:28.270105 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:28.270105 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:28.270105 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:28.270105 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:28.270105 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:28.270105 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:28.270105 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:28.270105 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:49:28.271007 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:49:28.271007 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:28.271007 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:49:28.271007 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 198/394] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:49:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:28.271333 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56895616-b809-4bc3-ba8a-294dc8ef36ab None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:28.279603 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:28.296283 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5951aeaf-9108-4d27-860e-53f31fe2146c None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:28.329990 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98be9ba7-849d-4c45-9836-ce7aca59643f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:28.330645 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98be9ba7-849d-4c45-9836-ce7aca59643f None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:28.331058 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98be9ba7-849d-4c45-9836-ce7aca59643f None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:28.331333 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98be9ba7-849d-4c45-9836-ce7aca59643f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:28.341701 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98be9ba7-849d-4c45-9836-ce7aca59643f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:28.343204 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98be9ba7-849d-4c45-9836-ce7aca59643f None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:28.356956 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-98be9ba7-849d-4c45-9836-ce7aca59643f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:28.356956 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:28.356956 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:28.356956 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:28.356956 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:28.356956 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:28.356956 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:28.356956 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:28.356956 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:28.356956 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:28.356956 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:28.356956 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:28.356956 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:49:28.356956 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:49:28.356956 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:49:28.356956 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:49:28.358077 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:28.358077 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:28.358077 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:28.358077 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:28.358077 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:28.358077 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:28.358077 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:28.358077 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:28.358077 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:28.358077 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:28.358077 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:28.358077 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:28.358077 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:28.358077 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:28.358077 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:28.358077 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:28.358077 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:49:28.358953 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:49:28.358953 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:28.358953 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:28.358953 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 197/395] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:49:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:28.358953 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98be9ba7-849d-4c45-9836-ce7aca59643f None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:28.371832 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56895616-b809-4bc3-ba8a-294dc8ef36ab None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:28.404785 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ddfb1f-bdf3-482b-bc8d-e896fe477ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:28.405389 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ddfb1f-bdf3-482b-bc8d-e896fe477ae6 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:28.405672 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ddfb1f-bdf3-482b-bc8d-e896fe477ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:28.405993 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ddfb1f-bdf3-482b-bc8d-e896fe477ae6 None admin] PATH_INFO: `/v3/domains` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:28.406642 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ddfb1f-bdf3-482b-bc8d-e896fe477ae6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:28.406950 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ddfb1f-bdf3-482b-bc8d-e896fe477ae6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:28.408385 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ddfb1f-bdf3-482b-bc8d-e896fe477ae6 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:28.414795 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:28.419747 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 199/396] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Oct 14 02:49:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:28.420531 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ddfb1f-bdf3-482b-bc8d-e896fe477ae6 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:28.455415 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98be9ba7-849d-4c45-9836-ce7aca59643f None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:28.485584 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:28.488151 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18effa74-fb0f-439e-bc75-6f2150a6bb79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:28.488697 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18effa74-fb0f-439e-bc75-6f2150a6bb79 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:28.488977 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18effa74-fb0f-439e-bc75-6f2150a6bb79 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:28.489736 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18effa74-fb0f-439e-bc75-6f2150a6bb79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:28.500335 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18effa74-fb0f-439e-bc75-6f2150a6bb79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:28.501811 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18effa74-fb0f-439e-bc75-6f2150a6bb79 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:28.516649 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-18effa74-fb0f-439e-bc75-6f2150a6bb79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:28.516649 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:28.516649 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:28.516649 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:28.516649 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:28.516649 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:28.516649 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:28.516649 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:28.516649 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:28.516649 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:28.516649 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:28.516649 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:28.516649 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:49:28.516649 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:49:28.516649 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:49:28.516649 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:49:28.517699 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:28.517699 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:28.517699 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:28.517699 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:28.517699 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:28.517699 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:28.517699 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:28.517699 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:28.517699 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:28.517699 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:28.517699 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:28.517699 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:28.517699 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:28.517699 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:28.517699 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:28.517699 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:28.517699 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:49:28.518589 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:49:28.518589 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:28.518589 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:28.518589 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 198/397] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:49:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:28.518589 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18effa74-fb0f-439e-bc75-6f2150a6bb79 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:28.525332 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ddfb1f-bdf3-482b-bc8d-e896fe477ae6 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:28.558077 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f986c178-d2a3-4a95-8c22-88e62dbaab3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:28.558676 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f986c178-d2a3-4a95-8c22-88e62dbaab3c None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:28.558960 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f986c178-d2a3-4a95-8c22-88e62dbaab3c None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:28.559234 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f986c178-d2a3-4a95-8c22-88e62dbaab3c None admin] PATH_INFO: `/v3/domains` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:28.559870 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f986c178-d2a3-4a95-8c22-88e62dbaab3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:28.560174 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f986c178-d2a3-4a95-8c22-88e62dbaab3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:28.561559 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f986c178-d2a3-4a95-8c22-88e62dbaab3c None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:28.573411 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 200/398] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Oct 14 02:49:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:28.574496 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f986c178-d2a3-4a95-8c22-88e62dbaab3c None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:28.615490 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18effa74-fb0f-439e-bc75-6f2150a6bb79 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:28.627554 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:28.647848 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a83fe8-e23d-4353-8f6b-4d95470532a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:28.648446 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a83fe8-e23d-4353-8f6b-4d95470532a3 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:28.648714 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a83fe8-e23d-4353-8f6b-4d95470532a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:28.648985 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a83fe8-e23d-4353-8f6b-4d95470532a3 None admin] PATH_INFO: `/v3/domains` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:28.649619 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a83fe8-e23d-4353-8f6b-4d95470532a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:28.649950 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a83fe8-e23d-4353-8f6b-4d95470532a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:28.651317 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a83fe8-e23d-4353-8f6b-4d95470532a3 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:28.663690 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 199/399] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Oct 14 02:49:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:28.664912 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a83fe8-e23d-4353-8f6b-4d95470532a3 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:28.676420 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f986c178-d2a3-4a95-8c22-88e62dbaab3c None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:28.701312 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:28.710023 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94370c9-71a9-4ef7-8938-f2bcb43c51ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:28.712181 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94370c9-71a9-4ef7-8938-f2bcb43c51ed None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:28.712181 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94370c9-71a9-4ef7-8938-f2bcb43c51ed None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:28.712181 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94370c9-71a9-4ef7-8938-f2bcb43c51ed None admin] PATH_INFO: `/v3/domains` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:28.712181 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94370c9-71a9-4ef7-8938-f2bcb43c51ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:28.712181 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94370c9-71a9-4ef7-8938-f2bcb43c51ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:28.713495 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94370c9-71a9-4ef7-8938-f2bcb43c51ed None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:28.724857 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 201/400] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Oct 14 02:49:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:28.725863 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94370c9-71a9-4ef7-8938-f2bcb43c51ed None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:28.765394 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a83fe8-e23d-4353-8f6b-4d95470532a3 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:28.798150 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64795680-45c0-446c-97c9-71f66024015a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:28.798699 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64795680-45c0-446c-97c9-71f66024015a None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:28.798975 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64795680-45c0-446c-97c9-71f66024015a None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:28.799244 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64795680-45c0-446c-97c9-71f66024015a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:28.806474 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64795680-45c0-446c-97c9-71f66024015a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:28.807894 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64795680-45c0-446c-97c9-71f66024015a None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:28.819065 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-64795680-45c0-446c-97c9-71f66024015a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 14 02:49:28.819065 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:28.819065 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:28.819065 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:28.819065 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:28.819065 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:28.819065 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:28.819065 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:28.819065 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:28.819065 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:28.819065 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:28.819065 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:28.819065 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 14 02:49:28.819065 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 14 02:49:28.819065 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 14 02:49:28.819065 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 14 02:49:28.820077 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:28.820077 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:28.820077 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 14 02:49:28.820077 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:49:28.820077 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 14 02:49:28.820077 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:49:28.820077 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:28.820077 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:28.820077 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:28.820077 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:28.820077 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:28.820077 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:28.820077 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:28.820077 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:28.820077 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:28.820077 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:28.820077 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:28.820077 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:28.820977 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:28.820977 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:28.820977 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 14 02:49:28.820977 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 14 02:49:28.820977 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 14 02:49:28.820977 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 14 02:49:28.820977 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 14 02:49:28.820977 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 14 02:49:28.820977 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 14 02:49:28.820977 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:28.820977 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 200/401] 10.222.0.39 () {56 vars in 1102 bytes} [Fri Oct 14 02:49:28 2022] GET /identity/v3/users/neutron => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:28.820977 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64795680-45c0-446c-97c9-71f66024015a None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:28.827817 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94370c9-71a9-4ef7-8938-f2bcb43c51ed None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:28.850496 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:28.861005 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff680b3-2333-44aa-8de1-65d07799fef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:28.861598 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff680b3-2333-44aa-8de1-65d07799fef0 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:28.862004 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff680b3-2333-44aa-8de1-65d07799fef0 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:28.862235 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff680b3-2333-44aa-8de1-65d07799fef0 None admin] PATH_INFO: `/v3/domains` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:28.879062 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff680b3-2333-44aa-8de1-65d07799fef0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:28.879062 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff680b3-2333-44aa-8de1-65d07799fef0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:28.879062 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff680b3-2333-44aa-8de1-65d07799fef0 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:28.879062 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 202/402] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Oct 14 02:49:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:28.879062 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff680b3-2333-44aa-8de1-65d07799fef0 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:28.916185 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64795680-45c0-446c-97c9-71f66024015a None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:28.924193 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:28.948639 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a763420-daf3-4429-be71-2501514f14df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:28.949236 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a763420-daf3-4429-be71-2501514f14df None admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:28.949526 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a763420-daf3-4429-be71-2501514f14df None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:28.949845 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a763420-daf3-4429-be71-2501514f14df None admin] PATH_INFO: `/v3/users` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:28.950490 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a763420-daf3-4429-be71-2501514f14df None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:28.951925 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a763420-daf3-4429-be71-2501514f14df None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:28.975693 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff680b3-2333-44aa-8de1-65d07799fef0 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:29.008923 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6807ffe8-54db-4401-a6b2-fc6aa6cb44b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:29.009608 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6807ffe8-54db-4401-a6b2-fc6aa6cb44b0 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:29.009870 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6807ffe8-54db-4401-a6b2-fc6aa6cb44b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:29.010149 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6807ffe8-54db-4401-a6b2-fc6aa6cb44b0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:29.016603 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6807ffe8-54db-4401-a6b2-fc6aa6cb44b0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:29.018093 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6807ffe8-54db-4401-a6b2-fc6aa6cb44b0 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:29.030187 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-6807ffe8-54db-4401-a6b2-fc6aa6cb44b0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:49:29.030187 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:29.030187 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:29.030187 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:29.030187 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:29.030187 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:29.030187 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:29.030187 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:29.030187 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:29.030187 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:29.030187 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:29.030187 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:29.030187 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 14 02:49:29.030187 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 14 02:49:29.030187 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 14 02:49:29.031122 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 14 02:49:29.031122 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:29.031122 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:29.031122 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:29.031122 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:29.031122 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:29.031122 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:29.031122 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:29.031122 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:29.031122 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:29.031122 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:29.031122 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:29.031122 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:29.031122 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:29.031122 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:29.031122 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:29.031122 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:29.037300 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 14 02:49:29.037300 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 14 02:49:29.037300 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 14 02:49:29.037300 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 14 02:49:29.037300 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 14 02:49:29.037300 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 14 02:49:29.037300 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:49:29.037300 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:49:29.037300 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 203/403] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Oct 14 02:49:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:29.037300 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6807ffe8-54db-4401-a6b2-fc6aa6cb44b0 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:29.043211 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 201/404] 10.222.0.39 () {60 vars in 1132 bytes} [Fri Oct 14 02:49:28 2022] POST /identity/v3/users => generated 247 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:49:29.043211 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a763420-daf3-4429-be71-2501514f14df None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:29.143390 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:29.157489 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a763420-daf3-4429-be71-2501514f14df None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:29.162867 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6807ffe8-54db-4401-a6b2-fc6aa6cb44b0 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:29.171168 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:29.195258 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dec775-750b-417c-95c9-3e470f1625fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:29.196110 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dec775-750b-417c-95c9-3e470f1625fd None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:29.196396 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dec775-750b-417c-95c9-3e470f1625fd None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:29.196669 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dec775-750b-417c-95c9-3e470f1625fd None admin] PATH_INFO: `/v3/users` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:29.198085 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dec775-750b-417c-95c9-3e470f1625fd None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:29.198453 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dec775-750b-417c-95c9-3e470f1625fd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:29.200156 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dec775-750b-417c-95c9-3e470f1625fd None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:29.202076 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03504037-b312-447e-a247-8ca3f874af26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:29.202721 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03504037-b312-447e-a247-8ca3f874af26 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:29.203032 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03504037-b312-447e-a247-8ca3f874af26 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:29.203310 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03504037-b312-447e-a247-8ca3f874af26 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:29.211565 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03504037-b312-447e-a247-8ca3f874af26 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:29.213071 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03504037-b312-447e-a247-8ca3f874af26 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:29.221237 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 202/405] 10.222.0.39 () {56 vars in 1139 bytes} [Fri Oct 14 02:49:29 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:29.222299 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dec775-750b-417c-95c9-3e470f1625fd None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:29.225421 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-03504037-b312-447e-a247-8ca3f874af26 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:49:29.225421 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:29.225421 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:29.225421 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:29.225421 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:29.225421 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:29.225421 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:29.225421 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:29.225421 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:29.225421 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:29.225421 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:29.225421 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:29.225421 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 14 02:49:29.225421 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 14 02:49:29.225421 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 14 02:49:29.226437 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 14 02:49:29.226437 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:29.226437 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:29.226437 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:29.226437 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:29.226437 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:29.226437 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:29.226437 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:29.226437 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:29.226437 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:29.226437 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:29.226437 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:29.226437 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:29.226437 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:29.226437 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:29.226437 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:29.226437 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:29.227409 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 14 02:49:29.227409 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 14 02:49:29.227409 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 14 02:49:29.227409 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 14 02:49:29.227409 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 14 02:49:29.227409 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 14 02:49:29.227409 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:49:29.227409 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:49:29.227409 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 204/406] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Oct 14 02:49:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:29.227409 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03504037-b312-447e-a247-8ca3f874af26 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:29.333448 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dec775-750b-417c-95c9-3e470f1625fd None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:29.342368 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03504037-b312-447e-a247-8ca3f874af26 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:29.367199 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8778ee-e1ca-42f7-a057-f5553c27571f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:29.367916 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8778ee-e1ca-42f7-a057-f5553c27571f None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:29.368207 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8778ee-e1ca-42f7-a057-f5553c27571f None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:29.368479 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8778ee-e1ca-42f7-a057-f5553c27571f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:29.374988 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8778ee-e1ca-42f7-a057-f5553c27571f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:29.376468 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8778ee-e1ca-42f7-a057-f5553c27571f None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:29.380884 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12155b55-1511-4a92-8485-8f6e5f89f522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:29.381663 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12155b55-1511-4a92-8485-8f6e5f89f522 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:29.382193 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12155b55-1511-4a92-8485-8f6e5f89f522 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:29.382664 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12155b55-1511-4a92-8485-8f6e5f89f522 None admin] PATH_INFO: `/v3/projects` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:29.383513 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12155b55-1511-4a92-8485-8f6e5f89f522 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:29.383965 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12155b55-1511-4a92-8485-8f6e5f89f522 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:29.385535 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12155b55-1511-4a92-8485-8f6e5f89f522 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:29.389325 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-6d8778ee-e1ca-42f7-a057-f5553c27571f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:49:29.389325 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:29.389325 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:29.389325 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:29.389325 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:29.389325 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:29.389325 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:29.389325 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:29.389325 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:29.389325 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:29.389325 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:29.389325 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:29.389325 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 14 02:49:29.389325 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 14 02:49:29.389325 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 14 02:49:29.390397 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 14 02:49:29.390397 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:29.390397 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:29.390397 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:29.390397 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:29.390397 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:29.390397 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:29.390397 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:29.390397 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:29.390397 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:29.390397 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:29.390397 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:29.390397 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:29.390397 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:29.390397 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:29.390397 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:29.390397 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:29.391244 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 14 02:49:29.391244 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 14 02:49:29.391244 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 14 02:49:29.391244 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 14 02:49:29.391244 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 14 02:49:29.391244 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 14 02:49:29.391244 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:49:29.391244 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:29.391244 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 203/407] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Oct 14 02:49:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:29.391244 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8778ee-e1ca-42f7-a057-f5553c27571f None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:29.402422 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:29.411781 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 205/408] 10.222.0.39 () {56 vars in 1148 bytes} [Fri Oct 14 02:49:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:29.412840 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12155b55-1511-4a92-8485-8f6e5f89f522 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:29.443958 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:29.498122 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8778ee-e1ca-42f7-a057-f5553c27571f None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:29.520396 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12155b55-1511-4a92-8485-8f6e5f89f522 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:29.532308 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca96bdf-ba44-4ee2-946a-2a4cf47f3df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:29.532941 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca96bdf-ba44-4ee2-946a-2a4cf47f3df8 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:29.533222 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca96bdf-ba44-4ee2-946a-2a4cf47f3df8 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:29.533496 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca96bdf-ba44-4ee2-946a-2a4cf47f3df8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:29.545859 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca96bdf-ba44-4ee2-946a-2a4cf47f3df8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:29.547309 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca96bdf-ba44-4ee2-946a-2a4cf47f3df8 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:29.559016 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e9ce68-a33f-42d8-b3ba-f2d207a28ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:29.559696 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e9ce68-a33f-42d8-b3ba-f2d207a28ed3 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:29.560048 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e9ce68-a33f-42d8-b3ba-f2d207a28ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:29.560392 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e9ce68-a33f-42d8-b3ba-f2d207a28ed3 None admin] PATH_INFO: `/v3/projects` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:29.561153 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e9ce68-a33f-42d8-b3ba-f2d207a28ed3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:29.561478 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-bca96bdf-ba44-4ee2-946a-2a4cf47f3df8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:29.561478 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:29.561478 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:29.561478 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:29.561478 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:29.561478 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:29.561478 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:29.561478 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:29.561478 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:29.561478 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:29.561478 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:29.561478 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:29.561478 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:49:29.561478 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:49:29.561478 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:49:29.561478 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:49:29.562624 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:29.562624 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:29.562624 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:29.562624 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:29.562624 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:29.562624 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:29.562624 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:29.562624 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:29.562624 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:29.562624 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:29.562624 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:29.562624 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:29.562624 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:29.562624 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:29.562624 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:29.562624 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:29.562624 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:49:29.563842 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:49:29.563842 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:29.563842 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:29.564057 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e9ce68-a33f-42d8-b3ba-f2d207a28ed3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:29.564585 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e9ce68-a33f-42d8-b3ba-f2d207a28ed3 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:29.567784 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 204/409] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:49:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:29.568466 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca96bdf-ba44-4ee2-946a-2a4cf47f3df8 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:29.582601 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 206/410] 10.222.0.39 () {56 vars in 1148 bytes} [Fri Oct 14 02:49:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:29.582601 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e9ce68-a33f-42d8-b3ba-f2d207a28ed3 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:29.637352 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:29.671771 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:29.682002 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca96bdf-ba44-4ee2-946a-2a4cf47f3df8 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:29.687163 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e9ce68-a33f-42d8-b3ba-f2d207a28ed3 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:29.718289 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9a27df-46c5-4f55-b19c-ee93991127f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:29.718901 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a27df-46c5-4f55-b19c-ee93991127f3 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:29.719183 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a27df-46c5-4f55-b19c-ee93991127f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:29.719459 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a27df-46c5-4f55-b19c-ee93991127f3 None admin] PATH_INFO: `/v3/projects` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:29.720151 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9a27df-46c5-4f55-b19c-ee93991127f3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:29.720458 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9a27df-46c5-4f55-b19c-ee93991127f3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:29.721918 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9a27df-46c5-4f55-b19c-ee93991127f3 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:29.729528 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a0f471-ff83-45cc-a92b-621e389850e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:29.730402 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a0f471-ff83-45cc-a92b-621e389850e8 None admin] REQUEST_METHOD: `PUT` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:29.730832 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a0f471-ff83-45cc-a92b-621e389850e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:29.731255 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a0f471-ff83-45cc-a92b-621e389850e8 None admin] PATH_INFO: `/v3/projects/dac9f8cd285c44e889a621b8b7fe479e/users/a6b07bf0faca47b28b50990d1720853f/roles/9c7226aa19e742759967212b7291782b` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:29.740174 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 205/411] 10.222.0.39 () {56 vars in 1148 bytes} [Fri Oct 14 02:49:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:29.741007 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9a27df-46c5-4f55-b19c-ee93991127f3 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:29.752183 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a0f471-ff83-45cc-a92b-621e389850e8 None admin] RBAC: Authorizing `identity:create_grant(project_id=dac9f8cd285c44e889a621b8b7fe479e, user_id=a6b07bf0faca47b28b50990d1720853f, role_id=9c7226aa19e742759967212b7291782b)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:29.754004 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a0f471-ff83-45cc-a92b-621e389850e8 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:29.819373 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 207/412] 10.222.0.39 () {58 vars in 1439 bytes} [Fri Oct 14 02:49:29 2022] PUT /identity/v3/projects/dac9f8cd285c44e889a621b8b7fe479e/users/a6b07bf0faca47b28b50990d1720853f/roles/9c7226aa19e742759967212b7291782b => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:49:29.820239 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a0f471-ff83-45cc-a92b-621e389850e8 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:29.859958 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9a27df-46c5-4f55-b19c-ee93991127f3 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:29.882592 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:29.903175 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3838dd8-3b8a-4029-bdf9-e10d5a28f7b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:29.903805 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3838dd8-3b8a-4029-bdf9-e10d5a28f7b8 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:29.904089 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3838dd8-3b8a-4029-bdf9-e10d5a28f7b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:29.904365 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3838dd8-3b8a-4029-bdf9-e10d5a28f7b8 None admin] PATH_INFO: `/v3/domains` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:29.905030 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3838dd8-3b8a-4029-bdf9-e10d5a28f7b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:29.905396 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3838dd8-3b8a-4029-bdf9-e10d5a28f7b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:29.906904 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3838dd8-3b8a-4029-bdf9-e10d5a28f7b8 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:29.921900 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 206/413] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Oct 14 02:49:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:29.922674 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3838dd8-3b8a-4029-bdf9-e10d5a28f7b8 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:29.982470 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a0f471-ff83-45cc-a92b-621e389850e8 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:30.038532 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a534f13-3f33-40e3-be79-c4b19e0801a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:30.039201 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a534f13-3f33-40e3-be79-c4b19e0801a9 None admin] REQUEST_METHOD: `PUT` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:30.039477 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a534f13-3f33-40e3-be79-c4b19e0801a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:30.039746 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a534f13-3f33-40e3-be79-c4b19e0801a9 None admin] PATH_INFO: `/v3/projects/dac9f8cd285c44e889a621b8b7fe479e/users/6a3cd1fbe6df48f8aa90d4ea69905e9e/roles/9c7226aa19e742759967212b7291782b` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:30.042065 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3838dd8-3b8a-4029-bdf9-e10d5a28f7b8 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:30.045017 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:30.058948 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a534f13-3f33-40e3-be79-c4b19e0801a9 None admin] RBAC: Authorizing `identity:create_grant(project_id=dac9f8cd285c44e889a621b8b7fe479e, user_id=6a3cd1fbe6df48f8aa90d4ea69905e9e, role_id=9c7226aa19e742759967212b7291782b)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:30.060418 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a534f13-3f33-40e3-be79-c4b19e0801a9 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:30.087540 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0c2fe2-d74f-4110-823e-4aab20075dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:30.088203 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0c2fe2-d74f-4110-823e-4aab20075dba None admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:30.088487 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0c2fe2-d74f-4110-823e-4aab20075dba None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:30.088760 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0c2fe2-d74f-4110-823e-4aab20075dba None admin] PATH_INFO: `/v3/projects/dac9f8cd285c44e889a621b8b7fe479e/users/ab599d98b0624a2e89090312bcab713f/roles/9c7226aa19e742759967212b7291782b` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:30.103893 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0c2fe2-d74f-4110-823e-4aab20075dba None admin] RBAC: Authorizing `identity:create_grant(project_id=dac9f8cd285c44e889a621b8b7fe479e, user_id=ab599d98b0624a2e89090312bcab713f, role_id=9c7226aa19e742759967212b7291782b)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:30.107190 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0c2fe2-d74f-4110-823e-4aab20075dba None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:30.133172 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 208/414] 10.222.0.39 () {58 vars in 1439 bytes} [Fri Oct 14 02:49:29 2022] PUT /identity/v3/projects/dac9f8cd285c44e889a621b8b7fe479e/users/6a3cd1fbe6df48f8aa90d4ea69905e9e/roles/9c7226aa19e742759967212b7291782b => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:49:30.137807 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a534f13-3f33-40e3-be79-c4b19e0801a9 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:30.220324 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 207/415] 10.222.0.39 () {58 vars in 1439 bytes} [Fri Oct 14 02:49:29 2022] PUT /identity/v3/projects/dac9f8cd285c44e889a621b8b7fe479e/users/ab599d98b0624a2e89090312bcab713f/roles/9c7226aa19e742759967212b7291782b => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:49:30.265255 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a534f13-3f33-40e3-be79-c4b19e0801a9 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:30.315709 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45a4966-4817-44b4-b398-2a2bd5c3f06a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:30.316371 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45a4966-4817-44b4-b398-2a2bd5c3f06a None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:30.316660 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45a4966-4817-44b4-b398-2a2bd5c3f06a None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:30.317011 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45a4966-4817-44b4-b398-2a2bd5c3f06a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:30.325030 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45a4966-4817-44b4-b398-2a2bd5c3f06a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:30.326560 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45a4966-4817-44b4-b398-2a2bd5c3f06a None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:30.343942 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-d45a4966-4817-44b4-b398-2a2bd5c3f06a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:49:30.343942 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:30.343942 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:30.343942 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:30.343942 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:30.343942 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:30.343942 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:30.343942 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:30.343942 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:30.343942 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:30.343942 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:30.343942 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:30.343942 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 14 02:49:30.343942 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 14 02:49:30.343942 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 14 02:49:30.344911 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 14 02:49:30.344911 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:30.344911 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:30.344911 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:30.344911 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:30.344911 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:30.344911 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:30.344911 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:30.344911 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:30.344911 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:30.344911 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:30.344911 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:30.344911 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:30.344911 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:30.344911 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:30.344911 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:30.344911 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:30.345753 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 14 02:49:30.345753 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 14 02:49:30.345753 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 14 02:49:30.345753 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 14 02:49:30.345753 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 14 02:49:30.345753 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 14 02:49:30.345753 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:49:30.345753 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:49:30.345753 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 209/416] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Oct 14 02:49:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:30.351627 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0c2fe2-d74f-4110-823e-4aab20075dba None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:30.397572 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:30.479233 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0c2fe2-d74f-4110-823e-4aab20075dba None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:30.516704 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b66f520-6072-4a78-aaf2-5e77cc24d504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:30.517674 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b66f520-6072-4a78-aaf2-5e77cc24d504 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:30.518182 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b66f520-6072-4a78-aaf2-5e77cc24d504 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:30.518675 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b66f520-6072-4a78-aaf2-5e77cc24d504 None admin] PATH_INFO: `/v3/projects` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:30.519554 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b66f520-6072-4a78-aaf2-5e77cc24d504 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:30.520055 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b66f520-6072-4a78-aaf2-5e77cc24d504 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:30.522237 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b66f520-6072-4a78-aaf2-5e77cc24d504 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:30.534469 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 208/417] 10.222.0.39 () {56 vars in 1148 bytes} [Fri Oct 14 02:49:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:30.540479 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45a4966-4817-44b4-b398-2a2bd5c3f06a None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:30.591952 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:30.657662 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45a4966-4817-44b4-b398-2a2bd5c3f06a None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:30.703619 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd650eb0-5922-4458-8161-a70d6c4fe7de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:30.704294 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd650eb0-5922-4458-8161-a70d6c4fe7de None admin] REQUEST_METHOD: `PUT` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:30.704582 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd650eb0-5922-4458-8161-a70d6c4fe7de None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:30.704862 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd650eb0-5922-4458-8161-a70d6c4fe7de None admin] PATH_INFO: `/v3/projects/dac9f8cd285c44e889a621b8b7fe479e/users/b123c412ac714b06929a4bd86eca71e0/roles/9c7226aa19e742759967212b7291782b` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:30.724867 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd650eb0-5922-4458-8161-a70d6c4fe7de None admin] RBAC: Authorizing `identity:create_grant(project_id=dac9f8cd285c44e889a621b8b7fe479e, user_id=b123c412ac714b06929a4bd86eca71e0, role_id=9c7226aa19e742759967212b7291782b)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:30.726364 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd650eb0-5922-4458-8161-a70d6c4fe7de None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:30.819721 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 210/418] 10.222.0.39 () {58 vars in 1439 bytes} [Fri Oct 14 02:49:30 2022] PUT /identity/v3/projects/dac9f8cd285c44e889a621b8b7fe479e/users/b123c412ac714b06929a4bd86eca71e0/roles/9c7226aa19e742759967212b7291782b => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 02:49:31.348072 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e024d-cb30-4f58-b7a9-5d38ae4dad90 None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:31.348627 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e024d-cb30-4f58-b7a9-5d38ae4dad90 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:31.348732 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e024d-cb30-4f58-b7a9-5d38ae4dad90 None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:31.350099 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 209/419] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:49:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:49:31.355787 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc2bcfc-97eb-44b9-94c7-59cfaf3d4934 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:31.356082 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc2bcfc-97eb-44b9-94c7-59cfaf3d4934 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:31.356361 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc2bcfc-97eb-44b9-94c7-59cfaf3d4934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:31.379261 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-3fc2bcfc-97eb-44b9-94c7-59cfaf3d4934 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:49:31.381071 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-3fc2bcfc-97eb-44b9-94c7-59cfaf3d4934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:49:31.464529 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 211/420] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:49:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:49:31.899254 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e024d-cb30-4f58-b7a9-5d38ae4dad90 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:32.000009 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:32.023029 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e024d-cb30-4f58-b7a9-5d38ae4dad90 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:32.071692 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8908b37-f108-4984-9c20-16ad0280de07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:32.072338 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8908b37-f108-4984-9c20-16ad0280de07 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:32.072617 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8908b37-f108-4984-9c20-16ad0280de07 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:32.072888 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8908b37-f108-4984-9c20-16ad0280de07 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:32.084277 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8908b37-f108-4984-9c20-16ad0280de07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:49:32.089111 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-b8908b37-f108-4984-9c20-16ad0280de07 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 14 02:49:32.089111 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:32.089111 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:32.089111 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:32.089111 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:32.089111 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:32.089111 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:32.089111 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:32.089111 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:32.089111 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:32.089111 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:32.089111 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:32.089111 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:49:32.089111 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:49:32.089111 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:49:32.089111 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:49:32.090038 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:49:32.090038 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:49:32.090038 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:49:32.090038 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:49:32.090038 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:32.090038 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:32.090038 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:32.090038 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:32.090038 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:32.090038 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:32.090038 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:32.090038 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:32.090038 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:32.090038 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:32.090038 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:32.090038 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:32.090872 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:32.090872 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:32.090872 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:32.090872 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:32.090872 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:49:32.090872 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:49:32.090872 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:49:32.090872 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:49:32.090872 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:49:32.090872 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:49:32.090872 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 14 02:49:32.090872 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:32.090872 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 210/421] 10.222.0.39 () {56 vars in 1102 bytes} [Fri Oct 14 02:49:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:32.096215 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc2bcfc-97eb-44b9-94c7-59cfaf3d4934 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:32.130470 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:32.225039 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc2bcfc-97eb-44b9-94c7-59cfaf3d4934 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:32.271463 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135dabd7-69c7-4913-addd-a7d162732920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:32.272076 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135dabd7-69c7-4913-addd-a7d162732920 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:32.272355 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135dabd7-69c7-4913-addd-a7d162732920 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:32.272627 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135dabd7-69c7-4913-addd-a7d162732920 None admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:32.273295 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135dabd7-69c7-4913-addd-a7d162732920 None admin] RBAC: Adding query filter params (name=service) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:32.273591 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135dabd7-69c7-4913-addd-a7d162732920 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:32.275005 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135dabd7-69c7-4913-addd-a7d162732920 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:32.283009 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 212/422] 10.222.0.39 () {56 vars in 1103 bytes} [Fri Oct 14 02:49:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:32.289440 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8908b37-f108-4984-9c20-16ad0280de07 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:32.435711 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8908b37-f108-4984-9c20-16ad0280de07 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:32.484973 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34d6a65-02e2-4782-9acf-08f19507f082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:32.485532 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34d6a65-02e2-4782-9acf-08f19507f082 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:32.485841 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34d6a65-02e2-4782-9acf-08f19507f082 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:32.486116 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34d6a65-02e2-4782-9acf-08f19507f082 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:32.497365 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d3f552-e482-44e7-93a8-28916804d563 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:32.497651 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d3f552-e482-44e7-93a8-28916804d563 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:32.497961 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d3f552-e482-44e7-93a8-28916804d563 None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:32.499973 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 213/423] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:49:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:49:32.502392 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34d6a65-02e2-4782-9acf-08f19507f082 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:32.503855 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34d6a65-02e2-4782-9acf-08f19507f082 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:32.506218 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fc3ee7-c6cb-4aa9-8c6f-aedd47eab66f None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:32.506666 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fc3ee7-c6cb-4aa9-8c6f-aedd47eab66f None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:32.506943 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fc3ee7-c6cb-4aa9-8c6f-aedd47eab66f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:32.512965 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:32.519700 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-a34d6a65-02e2-4782-9acf-08f19507f082 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:32.519700 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:32.519700 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:32.519700 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:32.519700 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:32.519700 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:32.519700 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:32.519700 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:32.519700 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:32.519700 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:32.519700 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:32.519700 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:32.519700 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:49:32.519700 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:49:32.519700 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:49:32.519700 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:49:32.521315 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:32.521315 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:32.521315 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:32.521315 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:32.521315 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:32.521315 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:32.521315 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:32.521315 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:32.521315 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:32.521315 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:32.521315 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:32.521315 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:32.521315 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:32.521315 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:32.521315 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:32.521315 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:32.521315 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:49:32.522212 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:49:32.522212 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:32.522212 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:32.525871 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 211/424] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:49:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:32.527795 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34d6a65-02e2-4782-9acf-08f19507f082 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:32.548142 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-e0fc3ee7-c6cb-4aa9-8c6f-aedd47eab66f None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:49:32.549796 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-e0fc3ee7-c6cb-4aa9-8c6f-aedd47eab66f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:49:32.684116 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 214/425] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:49:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:49:32.686806 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34d6a65-02e2-4782-9acf-08f19507f082 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:32.728957 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3854ea-155c-43d2-b7fd-71b96c2890bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:32.730129 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3854ea-155c-43d2-b7fd-71b96c2890bc None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:32.730555 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3854ea-155c-43d2-b7fd-71b96c2890bc None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:32.731080 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3854ea-155c-43d2-b7fd-71b96c2890bc None admin] PATH_INFO: `/v3/domains` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:32.731892 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3854ea-155c-43d2-b7fd-71b96c2890bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:32.732348 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3854ea-155c-43d2-b7fd-71b96c2890bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:32.733957 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3854ea-155c-43d2-b7fd-71b96c2890bc None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:32.758664 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 212/426] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Oct 14 02:49:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:32.762331 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fc3ee7-c6cb-4aa9-8c6f-aedd47eab66f None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:32.794860 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:32.907165 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8a76bc-0226-41a4-a4b0-1d01f8774f8c None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:32.907487 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8a76bc-0226-41a4-a4b0-1d01f8774f8c None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:32.907720 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8a76bc-0226-41a4-a4b0-1d01f8774f8c None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:32.909312 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 213/427] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:49:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:49:32.914834 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b95eb1d-5f82-4ddb-aaed-0f326c6b83f4 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:32.915122 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b95eb1d-5f82-4ddb-aaed-0f326c6b83f4 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:32.915384 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b95eb1d-5f82-4ddb-aaed-0f326c6b83f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:32.929234 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fc3ee7-c6cb-4aa9-8c6f-aedd47eab66f None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:32.935510 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-6b95eb1d-5f82-4ddb-aaed-0f326c6b83f4 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:49:32.936766 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-6b95eb1d-5f82-4ddb-aaed-0f326c6b83f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:49:32.989983 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b4e95d-8435-41c8-a138-c01c3077edcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:32.990625 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b4e95d-8435-41c8-a138-c01c3077edcd None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:32.990904 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b4e95d-8435-41c8-a138-c01c3077edcd None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:32.991172 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b4e95d-8435-41c8-a138-c01c3077edcd None admin] PATH_INFO: `/v3/users/swift` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:32.998692 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b4e95d-8435-41c8-a138-c01c3077edcd None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:33.000222 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b4e95d-8435-41c8-a138-c01c3077edcd None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:33.000541 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:33.017638 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 214/428] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:49:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:49:33.017638 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b95eb1d-5f82-4ddb-aaed-0f326c6b83f4 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:33.018560 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-92b4e95d-8435-41c8-a138-c01c3077edcd None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 14 02:49:33.018560 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:33.018560 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:33.018560 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:33.018560 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:33.018560 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:33.018560 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:33.018560 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:33.018560 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:33.018560 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:33.018560 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:33.018560 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:33.018560 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 14 02:49:33.018560 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 14 02:49:33.018560 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 14 02:49:33.018560 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 14 02:49:33.019363 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:33.019363 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:33.019363 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 14 02:49:33.019363 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:49:33.019363 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 14 02:49:33.019363 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:49:33.019363 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:33.019363 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:33.019363 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:33.019363 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:33.019363 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:33.019363 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:33.019363 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:33.019363 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:33.019363 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:33.019363 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:33.019363 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:33.019363 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:33.020275 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:33.020275 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:33.020275 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 14 02:49:33.020275 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 14 02:49:33.020275 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 14 02:49:33.020275 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 14 02:49:33.020275 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 14 02:49:33.020275 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 14 02:49:33.020275 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 14 02:49:33.020275 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:49:33.021788 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 215/429] 10.222.0.39 () {56 vars in 1096 bytes} [Fri Oct 14 02:49:32 2022] GET /identity/v3/users/swift => generated 83 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:33.027160 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b4e95d-8435-41c8-a138-c01c3077edcd None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:33.121715 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b95eb1d-5f82-4ddb-aaed-0f326c6b83f4 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:33.151647 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:33.163095 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d387d8c0-ab71-4446-8b06-4092fba5315c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:33.163740 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d387d8c0-ab71-4446-8b06-4092fba5315c None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:33.164015 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d387d8c0-ab71-4446-8b06-4092fba5315c None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:33.164284 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d387d8c0-ab71-4446-8b06-4092fba5315c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:33.174914 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d387d8c0-ab71-4446-8b06-4092fba5315c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:49:33.180266 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-d387d8c0-ab71-4446-8b06-4092fba5315c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 14 02:49:33.180266 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:33.180266 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:33.180266 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:33.180266 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:33.180266 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:33.180266 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:33.180266 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:33.180266 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:33.180266 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:33.180266 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:33.180266 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:33.180266 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:49:33.180266 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:49:33.180266 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:49:33.180266 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:49:33.181137 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:49:33.181137 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:49:33.181137 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:49:33.181137 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:49:33.181137 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:33.181137 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:33.181137 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:33.181137 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:33.181137 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:33.181137 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:33.181137 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:33.181137 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:33.181137 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:33.181137 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:33.181137 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:33.181137 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:33.182004 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:33.182004 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:33.182004 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:33.182004 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:33.182004 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:49:33.182004 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:49:33.182004 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:49:33.182004 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:49:33.182004 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:49:33.182004 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:49:33.182004 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 14 02:49:33.182004 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:33.182004 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 215/430] 10.222.0.39 () {56 vars in 1102 bytes} [Fri Oct 14 02:49:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:33.182663 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eff06f8-1628-4fa7-af90-828c40d0aacd None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:33.183514 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eff06f8-1628-4fa7-af90-828c40d0aacd None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:33.183514 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eff06f8-1628-4fa7-af90-828c40d0aacd None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:33.184501 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b4e95d-8435-41c8-a138-c01c3077edcd None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:33.193497 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 216/431] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:49:33 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:49:33.193497 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eff06f8-1628-4fa7-af90-828c40d0aacd None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:33.234102 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8d1b5f-243b-4f27-93d8-d37857fb2c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:33.234704 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8d1b5f-243b-4f27-93d8-d37857fb2c04 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:33.234990 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8d1b5f-243b-4f27-93d8-d37857fb2c04 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:33.235260 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8d1b5f-243b-4f27-93d8-d37857fb2c04 None admin] PATH_INFO: `/v3/users` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:33.236415 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8d1b5f-243b-4f27-93d8-d37857fb2c04 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:33.236846 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8d1b5f-243b-4f27-93d8-d37857fb2c04 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:33.238602 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8d1b5f-243b-4f27-93d8-d37857fb2c04 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:33.259845 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:33.260134 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 216/432] 10.222.0.39 () {56 vars in 1135 bytes} [Fri Oct 14 02:49:33 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:33.264884 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76a6e95-47d5-4b94-b0ec-97d101f7bf39 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:33.264884 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76a6e95-47d5-4b94-b0ec-97d101f7bf39 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:33.264884 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76a6e95-47d5-4b94-b0ec-97d101f7bf39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:33.283567 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-c76a6e95-47d5-4b94-b0ec-97d101f7bf39 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:49:33.284907 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-c76a6e95-47d5-4b94-b0ec-97d101f7bf39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:49:33.323256 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eff06f8-1628-4fa7-af90-828c40d0aacd None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:33.358613 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657b5ebe-f3c9-4d68-8e73-a8ced9678378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:33.359221 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657b5ebe-f3c9-4d68-8e73-a8ced9678378 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:33.359506 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657b5ebe-f3c9-4d68-8e73-a8ced9678378 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:33.359777 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657b5ebe-f3c9-4d68-8e73-a8ced9678378 None admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:33.360433 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657b5ebe-f3c9-4d68-8e73-a8ced9678378 None admin] RBAC: Adding query filter params (name=service) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:33.360741 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657b5ebe-f3c9-4d68-8e73-a8ced9678378 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:33.362151 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657b5ebe-f3c9-4d68-8e73-a8ced9678378 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:33.367865 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 217/433] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:49:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:49:33.368977 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76a6e95-47d5-4b94-b0ec-97d101f7bf39 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:33.377905 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 217/434] 10.222.0.39 () {56 vars in 1103 bytes} [Fri Oct 14 02:49:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:33.378939 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657b5ebe-f3c9-4d68-8e73-a8ced9678378 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:33.451898 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:33.491288 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76a6e95-47d5-4b94-b0ec-97d101f7bf39 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:33.514943 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657b5ebe-f3c9-4d68-8e73-a8ced9678378 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:33.530138 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8713d561-7a17-43fa-8501-80f01a5b8b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:33.531047 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8713d561-7a17-43fa-8501-80f01a5b8b7d None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:33.531555 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8713d561-7a17-43fa-8501-80f01a5b8b7d None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:33.532024 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8713d561-7a17-43fa-8501-80f01a5b8b7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:33.532903 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:33.543735 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8713d561-7a17-43fa-8501-80f01a5b8b7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:33.545492 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8713d561-7a17-43fa-8501-80f01a5b8b7d None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:33.559463 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a507c346-7cda-4a28-a3ee-49bc3a119045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:33.560101 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a507c346-7cda-4a28-a3ee-49bc3a119045 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:33.560380 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a507c346-7cda-4a28-a3ee-49bc3a119045 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:33.560652 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a507c346-7cda-4a28-a3ee-49bc3a119045 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:33.562265 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-8713d561-7a17-43fa-8501-80f01a5b8b7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:33.562265 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:33.562265 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:33.562265 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:33.562265 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:33.562265 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:33.562265 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:33.562265 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:33.562265 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:33.562265 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:33.562265 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:33.562265 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:33.562265 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:49:33.562265 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:49:33.562265 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:49:33.562265 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:49:33.563156 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:33.563156 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:33.563156 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:33.563156 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:33.563156 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:33.563156 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:33.563156 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:33.563156 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:33.563156 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:33.563156 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:33.563156 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:33.563156 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:33.563156 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:33.563156 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:33.563156 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:33.563156 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:33.563156 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:49:33.563988 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:49:33.563988 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:33.563988 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:49:33.570167 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 218/435] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:49:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:33.571224 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8713d561-7a17-43fa-8501-80f01a5b8b7d None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:33.571330 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a507c346-7cda-4a28-a3ee-49bc3a119045 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:49:33.577608 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-a507c346-7cda-4a28-a3ee-49bc3a119045 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 14 02:49:33.577608 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:33.577608 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:33.577608 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:33.577608 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:33.577608 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:33.577608 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:33.577608 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:33.577608 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:33.577608 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:33.577608 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:33.577608 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:33.577608 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:49:33.577608 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:49:33.577608 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:49:33.577608 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:49:33.578701 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:49:33.578701 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:49:33.578701 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:49:33.578701 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:49:33.578701 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:33.578701 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:33.578701 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:33.578701 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:33.578701 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:33.578701 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:33.578701 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:33.578701 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:33.578701 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:33.578701 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:33.578701 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:33.578701 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:33.579808 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:33.579808 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:33.579808 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:33.579808 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:33.579808 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:49:33.579808 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:49:33.579808 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:49:33.579808 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:49:33.579808 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:49:33.579808 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:49:33.579808 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 14 02:49:33.579808 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:33.579808 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 218/436] 10.222.0.39 () {56 vars in 1102 bytes} [Fri Oct 14 02:49:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:33.580719 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a507c346-7cda-4a28-a3ee-49bc3a119045 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:33.675508 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8713d561-7a17-43fa-8501-80f01a5b8b7d None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:33.690664 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a507c346-7cda-4a28-a3ee-49bc3a119045 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:33.691875 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:33.709361 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d548cf7-8789-47b7-9532-37d9473e71d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:33.710014 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d548cf7-8789-47b7-9532-37d9473e71d9 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:33.710301 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d548cf7-8789-47b7-9532-37d9473e71d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:33.710620 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d548cf7-8789-47b7-9532-37d9473e71d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:33.720735 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d548cf7-8789-47b7-9532-37d9473e71d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:33.722547 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d548cf7-8789-47b7-9532-37d9473e71d9 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:33.724420 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c299108-7bbb-42f7-a5f2-179c07fcbbf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:33.725021 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c299108-7bbb-42f7-a5f2-179c07fcbbf1 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:33.725293 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c299108-7bbb-42f7-a5f2-179c07fcbbf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:33.725559 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c299108-7bbb-42f7-a5f2-179c07fcbbf1 None admin] PATH_INFO: `/v3/domains` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:33.726220 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c299108-7bbb-42f7-a5f2-179c07fcbbf1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:33.726521 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c299108-7bbb-42f7-a5f2-179c07fcbbf1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:33.727870 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c299108-7bbb-42f7-a5f2-179c07fcbbf1 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:33.737380 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-9d548cf7-8789-47b7-9532-37d9473e71d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:33.737380 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:33.737380 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:33.737380 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:33.737380 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:33.737380 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:33.737380 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:33.737380 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:33.737380 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:33.737380 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:33.737380 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:33.737380 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:33.737380 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:49:33.737380 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:49:33.737380 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:49:33.737380 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:49:33.738323 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:33.738323 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:33.738323 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:33.738323 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:33.738323 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:33.738323 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:33.738323 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:33.738323 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:33.738323 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:33.738323 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:33.738323 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:33.738323 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:33.738323 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:33.738323 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:33.738323 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:33.738323 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:33.738323 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:49:33.739188 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:49:33.739188 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:33.739188 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:49:33.739188 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 219/437] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:49:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:33.739790 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 219/438] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Oct 14 02:49:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:33.740379 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c299108-7bbb-42f7-a5f2-179c07fcbbf1 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:33.743468 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840159c9-fc99-47a4-9268-eddcbbadfd54 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:33.743768 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840159c9-fc99-47a4-9268-eddcbbadfd54 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:33.744880 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840159c9-fc99-47a4-9268-eddcbbadfd54 None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:33.746252 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 220/439] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:49:33 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:49:33.746972 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840159c9-fc99-47a4-9268-eddcbbadfd54 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:33.771728 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:33.840135 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c299108-7bbb-42f7-a5f2-179c07fcbbf1 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:33.848175 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840159c9-fc99-47a4-9268-eddcbbadfd54 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:33.873584 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf04404-8c76-4eb0-b254-db07e7716699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:33.874210 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf04404-8c76-4eb0-b254-db07e7716699 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:33.874492 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf04404-8c76-4eb0-b254-db07e7716699 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:33.874766 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf04404-8c76-4eb0-b254-db07e7716699 None admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:33.875417 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf04404-8c76-4eb0-b254-db07e7716699 None admin] RBAC: Adding query filter params (name=service) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:33.875723 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf04404-8c76-4eb0-b254-db07e7716699 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:33.877089 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf04404-8c76-4eb0-b254-db07e7716699 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:33.883517 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700e666d-56c6-4733-998a-fd964ba1c9d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:33.884138 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700e666d-56c6-4733-998a-fd964ba1c9d0 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:33.884415 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700e666d-56c6-4733-998a-fd964ba1c9d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:33.884687 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700e666d-56c6-4733-998a-fd964ba1c9d0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:33.885152 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 220/440] 10.222.0.39 () {56 vars in 1103 bytes} [Fri Oct 14 02:49:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:33.885941 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf04404-8c76-4eb0-b254-db07e7716699 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:33.894780 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700e666d-56c6-4733-998a-fd964ba1c9d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:49:33.900493 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-700e666d-56c6-4733-998a-fd964ba1c9d0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 14 02:49:33.900493 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:33.900493 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:33.900493 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:33.900493 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:33.900493 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:33.900493 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:33.900493 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:33.900493 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:33.900493 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:33.900493 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:33.900493 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:33.900493 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:49:33.900493 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:49:33.900493 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:49:33.900493 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:49:33.901384 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:49:33.901384 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:49:33.901384 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:49:33.901384 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:49:33.901384 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:33.901384 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:33.901384 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:33.901384 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:33.901384 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:33.901384 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:33.901384 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:33.901384 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:33.901384 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:33.901384 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:33.901384 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:33.901384 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:33.902458 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:33.902458 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:33.902458 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:33.902458 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:33.902458 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:49:33.902458 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:49:33.902458 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:49:33.902458 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:49:33.902458 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:49:33.902458 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:49:33.902458 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 14 02:49:33.902458 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:49:33.902458 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 221/441] 10.222.0.39 () {56 vars in 1102 bytes} [Fri Oct 14 02:49:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:33.903377 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700e666d-56c6-4733-998a-fd964ba1c9d0 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:33.931834 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:33.981094 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:33.987674 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf04404-8c76-4eb0-b254-db07e7716699 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:33.998905 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700e666d-56c6-4733-998a-fd964ba1c9d0 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:34.023602 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0092aae-3296-47e0-857c-e6c71ee61150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:34.024208 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0092aae-3296-47e0-857c-e6c71ee61150 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:34.024487 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0092aae-3296-47e0-857c-e6c71ee61150 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:34.024759 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0092aae-3296-47e0-857c-e6c71ee61150 None admin] PATH_INFO: `/v3/domains` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:34.025388 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0092aae-3296-47e0-857c-e6c71ee61150 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:34.025689 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0092aae-3296-47e0-857c-e6c71ee61150 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:34.027092 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0092aae-3296-47e0-857c-e6c71ee61150 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:34.033982 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0f7e8c-898d-4523-bb9a-2bf35f10b7eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:34.034591 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0f7e8c-898d-4523-bb9a-2bf35f10b7eb None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:34.034869 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0f7e8c-898d-4523-bb9a-2bf35f10b7eb None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:34.035145 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0f7e8c-898d-4523-bb9a-2bf35f10b7eb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:34.038489 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 221/442] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Oct 14 02:49:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:34.039708 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eb5d0d-28df-4278-84df-fb5890c18fac None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:34.040000 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eb5d0d-28df-4278-84df-fb5890c18fac None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:34.040292 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eb5d0d-28df-4278-84df-fb5890c18fac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:34.041626 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0f7e8c-898d-4523-bb9a-2bf35f10b7eb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:34.043102 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0f7e8c-898d-4523-bb9a-2bf35f10b7eb None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:34.054265 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-2c0f7e8c-898d-4523-bb9a-2bf35f10b7eb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:49:34.054265 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:34.054265 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:34.054265 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:34.054265 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:34.054265 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:34.054265 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:34.054265 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:34.054265 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:34.054265 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:34.054265 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:34.054265 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:34.054265 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 14 02:49:34.054265 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 14 02:49:34.054265 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 14 02:49:34.055085 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 14 02:49:34.055085 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:34.055085 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:34.055085 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:34.055085 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:34.055085 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:34.055085 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:34.055085 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:34.055085 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:34.055085 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:34.055085 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:34.055085 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:34.055085 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:34.055085 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:34.055085 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:34.055085 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:34.055085 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:34.055921 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 14 02:49:34.055921 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 14 02:49:34.055921 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 14 02:49:34.055921 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 14 02:49:34.055921 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 14 02:49:34.055921 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 14 02:49:34.055921 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:49:34.055921 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:49:34.055921 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 222/443] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Oct 14 02:49:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:34.056875 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0f7e8c-898d-4523-bb9a-2bf35f10b7eb None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:34.060787 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-a3eb5d0d-28df-4278-84df-fb5890c18fac None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:49:34.062018 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-a3eb5d0d-28df-4278-84df-fb5890c18fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:49:34.127305 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 222/444] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:49:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:49:34.127305 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eb5d0d-28df-4278-84df-fb5890c18fac None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:34.147612 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:34.156885 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0f7e8c-898d-4523-bb9a-2bf35f10b7eb None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:34.192794 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2457f793-0329-4b0a-a3e4-485720f6c61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:34.193417 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2457f793-0329-4b0a-a3e4-485720f6c61e None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:34.193699 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2457f793-0329-4b0a-a3e4-485720f6c61e None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:34.194005 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2457f793-0329-4b0a-a3e4-485720f6c61e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:34.204275 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2457f793-0329-4b0a-a3e4-485720f6c61e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:34.205785 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2457f793-0329-4b0a-a3e4-485720f6c61e None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:34.221119 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-2457f793-0329-4b0a-a3e4-485720f6c61e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:34.221119 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:34.221119 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:34.221119 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:34.221119 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:34.221119 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:34.221119 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:34.221119 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:34.221119 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:34.221119 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:34.221119 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:34.221119 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:34.221119 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:49:34.221119 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:49:34.221119 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:49:34.221119 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:49:34.222228 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:34.222228 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:34.222228 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:34.222228 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:34.222228 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:34.222228 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:34.222228 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:34.222228 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:34.222228 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:34.222228 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:34.222228 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:34.222228 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:34.222228 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:34.222228 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:34.222228 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:34.222228 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:34.222228 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:49:34.223400 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:49:34.223400 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:34.223400 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:49:34.226881 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 223/445] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:49:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:34.227564 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2457f793-0329-4b0a-a3e4-485720f6c61e None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:34.227945 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:34.245668 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eb5d0d-28df-4278-84df-fb5890c18fac None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:34.281007 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e435b6-90f3-4449-b75c-a86ed943bc6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:34.281613 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e435b6-90f3-4449-b75c-a86ed943bc6c None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:34.281920 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e435b6-90f3-4449-b75c-a86ed943bc6c None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:34.282198 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e435b6-90f3-4449-b75c-a86ed943bc6c None admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:34.282854 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e435b6-90f3-4449-b75c-a86ed943bc6c None admin] RBAC: Adding query filter params (name=service) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:34.283158 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e435b6-90f3-4449-b75c-a86ed943bc6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:34.284530 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e435b6-90f3-4449-b75c-a86ed943bc6c None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:34.293806 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 223/446] 10.222.0.39 () {56 vars in 1103 bytes} [Fri Oct 14 02:49:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:34.294054 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e435b6-90f3-4449-b75c-a86ed943bc6c None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:34.338301 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2457f793-0329-4b0a-a3e4-485720f6c61e None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:34.380314 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2cfd17-4d01-4b7a-b8cd-10268a99bcd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:34.381246 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2cfd17-4d01-4b7a-b8cd-10268a99bcd5 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:34.381712 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2cfd17-4d01-4b7a-b8cd-10268a99bcd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:34.382195 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2cfd17-4d01-4b7a-b8cd-10268a99bcd5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:34.389782 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2cfd17-4d01-4b7a-b8cd-10268a99bcd5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:34.391400 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2cfd17-4d01-4b7a-b8cd-10268a99bcd5 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:34.400533 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e435b6-90f3-4449-b75c-a86ed943bc6c None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:34.402683 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-8b2cfd17-4d01-4b7a-b8cd-10268a99bcd5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 14 02:49:34.402683 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:34.402683 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:34.402683 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:34.402683 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:34.402683 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:34.402683 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:34.402683 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:34.402683 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:34.402683 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:34.402683 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:34.402683 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:34.402683 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 14 02:49:34.402683 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 14 02:49:34.402683 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 14 02:49:34.402683 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 14 02:49:34.403739 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:34.403739 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:34.403739 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 14 02:49:34.403739 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:49:34.403739 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 14 02:49:34.403739 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:49:34.403739 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:34.403739 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:34.403739 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:34.403739 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:34.403739 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:34.403739 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:34.403739 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:34.403739 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:34.403739 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:34.403739 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:34.403739 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:34.403739 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:34.404654 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:34.404654 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:34.404654 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 14 02:49:34.404654 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 14 02:49:34.404654 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 14 02:49:34.404654 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 14 02:49:34.404654 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 14 02:49:34.404654 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 14 02:49:34.404654 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 14 02:49:34.404654 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:49:34.405152 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 224/447] 10.222.0.39 () {56 vars in 1093 bytes} [Fri Oct 14 02:49:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:34.405518 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2cfd17-4d01-4b7a-b8cd-10268a99bcd5 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:34.408141 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:34.436221 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efe0559-1a00-4cdd-b4fb-11d266106532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:34.436827 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efe0559-1a00-4cdd-b4fb-11d266106532 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:34.437106 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efe0559-1a00-4cdd-b4fb-11d266106532 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:34.437384 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efe0559-1a00-4cdd-b4fb-11d266106532 None admin] PATH_INFO: `/v3/projects` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:34.438112 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efe0559-1a00-4cdd-b4fb-11d266106532 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:34.438419 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efe0559-1a00-4cdd-b4fb-11d266106532 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:34.439882 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efe0559-1a00-4cdd-b4fb-11d266106532 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:34.452062 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 224/448] 10.222.0.39 () {56 vars in 1148 bytes} [Fri Oct 14 02:49:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:34.452856 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efe0559-1a00-4cdd-b4fb-11d266106532 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:34.461014 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:34.511006 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2cfd17-4d01-4b7a-b8cd-10268a99bcd5 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:34.543298 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b69c31-2160-40c8-838f-b86f4053c72b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:34.543897 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b69c31-2160-40c8-838f-b86f4053c72b None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:34.544175 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b69c31-2160-40c8-838f-b86f4053c72b None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:34.544449 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b69c31-2160-40c8-838f-b86f4053c72b None admin] PATH_INFO: `/v3/domains` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:34.545081 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b69c31-2160-40c8-838f-b86f4053c72b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:34.545386 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b69c31-2160-40c8-838f-b86f4053c72b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:34.546793 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b69c31-2160-40c8-838f-b86f4053c72b None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:34.552466 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efe0559-1a00-4cdd-b4fb-11d266106532 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:34.558995 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 225/449] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Oct 14 02:49:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:34.559791 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b69c31-2160-40c8-838f-b86f4053c72b None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:34.591280 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c87911c-3064-4c24-a963-54506ea10440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:34.591908 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c87911c-3064-4c24-a963-54506ea10440 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:34.592190 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c87911c-3064-4c24-a963-54506ea10440 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:34.592462 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c87911c-3064-4c24-a963-54506ea10440 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:34.602034 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c87911c-3064-4c24-a963-54506ea10440 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:34.603481 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c87911c-3064-4c24-a963-54506ea10440 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:34.618407 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-9c87911c-3064-4c24-a963-54506ea10440 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:34.618407 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:34.618407 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:34.618407 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:34.618407 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:34.618407 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:34.618407 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:34.618407 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:34.618407 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:34.618407 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:34.618407 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:34.618407 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:34.618407 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:49:34.618407 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:49:34.618407 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:49:34.618407 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:49:34.619461 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:34.619461 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:34.619461 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:34.619461 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:34.619461 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:34.619461 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:34.619461 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:34.619461 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:34.619461 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:34.619461 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:34.619461 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:34.619461 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:34.619461 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:34.619461 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:34.619461 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:34.619461 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:34.619461 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:49:34.624988 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:49:34.624988 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:34.624988 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:34.624988 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 225/450] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:49:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:34.624988 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c87911c-3064-4c24-a963-54506ea10440 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:34.626121 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:34.657400 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b69c31-2160-40c8-838f-b86f4053c72b None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:34.678295 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:34.692925 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25b01d3-f85a-45d3-9795-b9dd87899025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:34.693539 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25b01d3-f85a-45d3-9795-b9dd87899025 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:34.693845 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25b01d3-f85a-45d3-9795-b9dd87899025 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:34.694116 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25b01d3-f85a-45d3-9795-b9dd87899025 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:34.704047 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25b01d3-f85a-45d3-9795-b9dd87899025 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:49:34.708756 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-c25b01d3-f85a-45d3-9795-b9dd87899025 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 14 02:49:34.708756 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:34.708756 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:34.708756 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:34.708756 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:34.708756 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:34.708756 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:34.708756 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:34.708756 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:34.708756 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:34.708756 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:34.708756 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:34.708756 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:49:34.708756 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:49:34.708756 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:49:34.708756 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:49:34.709804 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:49:34.709804 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:49:34.709804 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:49:34.709804 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:49:34.709804 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:34.709804 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:34.709804 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:34.709804 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:34.709804 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:34.709804 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:34.709804 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:34.709804 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:34.709804 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:34.709804 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:34.709804 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:34.709804 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:34.710727 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:34.710727 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:34.710727 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:34.710727 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:34.710727 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:49:34.710727 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:49:34.710727 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:49:34.710727 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:49:34.710727 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:49:34.710727 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:49:34.710727 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 14 02:49:34.710727 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:49:34.710727 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 226/451] 10.222.0.39 () {56 vars in 1102 bytes} [Fri Oct 14 02:49:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:34.711528 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25b01d3-f85a-45d3-9795-b9dd87899025 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:34.722277 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c87911c-3064-4c24-a963-54506ea10440 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:34.756403 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7bf12e-b550-4611-8604-440366af1ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:34.757197 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7bf12e-b550-4611-8604-440366af1ffb None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:34.757481 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7bf12e-b550-4611-8604-440366af1ffb None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:34.757781 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7bf12e-b550-4611-8604-440366af1ffb None admin] PATH_INFO: `/v3/users` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:34.758508 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7bf12e-b550-4611-8604-440366af1ffb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:34.758825 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7bf12e-b550-4611-8604-440366af1ffb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:34.760269 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7bf12e-b550-4611-8604-440366af1ffb None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:34.776997 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 226/452] 10.222.0.39 () {56 vars in 1133 bytes} [Fri Oct 14 02:49:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:34.777798 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7bf12e-b550-4611-8604-440366af1ffb None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:34.812049 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25b01d3-f85a-45d3-9795-b9dd87899025 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:34.833244 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:34.844619 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0ecfa6-d106-4e36-8b5e-cbf853f868d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:34.846621 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0ecfa6-d106-4e36-8b5e-cbf853f868d2 None admin] REQUEST_METHOD: `PUT` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:34.846621 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0ecfa6-d106-4e36-8b5e-cbf853f868d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:34.846621 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0ecfa6-d106-4e36-8b5e-cbf853f868d2 None admin] PATH_INFO: `/v3/projects/dac9f8cd285c44e889a621b8b7fe479e/users/5c5e07da64804243bc18862072b027a1/roles/9c7226aa19e742759967212b7291782b` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:34.860833 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0ecfa6-d106-4e36-8b5e-cbf853f868d2 None admin] RBAC: Authorizing `identity:create_grant(project_id=dac9f8cd285c44e889a621b8b7fe479e, user_id=5c5e07da64804243bc18862072b027a1, role_id=9c7226aa19e742759967212b7291782b)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:34.862355 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0ecfa6-d106-4e36-8b5e-cbf853f868d2 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:34.877411 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7bf12e-b550-4611-8604-440366af1ffb None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:34.893006 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:34.909659 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1442e255-e42e-45e4-b0ee-683728316f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:34.910249 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1442e255-e42e-45e4-b0ee-683728316f44 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:34.910528 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1442e255-e42e-45e4-b0ee-683728316f44 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:34.911312 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1442e255-e42e-45e4-b0ee-683728316f44 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:34.917950 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1442e255-e42e-45e4-b0ee-683728316f44 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:34.919481 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1442e255-e42e-45e4-b0ee-683728316f44 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:34.931260 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-1442e255-e42e-45e4-b0ee-683728316f44 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 14 02:49:34.931260 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:34.931260 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:34.931260 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:34.931260 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:34.931260 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:34.931260 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:34.931260 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:34.931260 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:34.931260 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:34.931260 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:34.931260 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:34.931260 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 14 02:49:34.931260 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 14 02:49:34.931260 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 14 02:49:34.931260 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 14 02:49:34.932367 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:34.932367 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:34.932367 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 14 02:49:34.932367 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:49:34.932367 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 14 02:49:34.932367 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:49:34.932367 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:34.932367 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:34.932367 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:34.932367 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:34.932367 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:34.932367 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:34.932367 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:34.932367 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:34.932367 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:34.932367 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:34.932367 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:34.932367 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:34.933459 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:34.933459 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:34.933459 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 14 02:49:34.933459 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 14 02:49:34.933459 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 14 02:49:34.933459 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 14 02:49:34.933459 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 14 02:49:34.933459 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 14 02:49:34.933459 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 14 02:49:34.933459 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:34.933459 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 227/453] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Oct 14 02:49:34 2022] GET /identity/v3/users/cinder => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:34.933459 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1442e255-e42e-45e4-b0ee-683728316f44 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:34.966433 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 227/454] 10.222.0.39 () {58 vars in 1439 bytes} [Fri Oct 14 02:49:34 2022] PUT /identity/v3/projects/dac9f8cd285c44e889a621b8b7fe479e/users/5c5e07da64804243bc18862072b027a1/roles/9c7226aa19e742759967212b7291782b => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:49:34.967290 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0ecfa6-d106-4e36-8b5e-cbf853f868d2 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:35.054867 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1442e255-e42e-45e4-b0ee-683728316f44 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:35.089473 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0ecfa6-d106-4e36-8b5e-cbf853f868d2 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:35.093395 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c90d503-1a3c-4d84-8723-a097db53a2e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:35.093395 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c90d503-1a3c-4d84-8723-a097db53a2e3 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:35.093395 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c90d503-1a3c-4d84-8723-a097db53a2e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:35.093810 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c90d503-1a3c-4d84-8723-a097db53a2e3 None admin] PATH_INFO: `/v3/domains` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:35.094346 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c90d503-1a3c-4d84-8723-a097db53a2e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:35.094650 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c90d503-1a3c-4d84-8723-a097db53a2e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:35.096062 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c90d503-1a3c-4d84-8723-a097db53a2e3 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:35.110052 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 228/455] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Oct 14 02:49:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:35.111045 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c90d503-1a3c-4d84-8723-a097db53a2e3 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:35.137544 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef5ef34-d123-416a-8819-59071c739f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:35.138202 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef5ef34-d123-416a-8819-59071c739f3a None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:35.138492 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef5ef34-d123-416a-8819-59071c739f3a None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:35.138767 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef5ef34-d123-416a-8819-59071c739f3a None admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:35.139437 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef5ef34-d123-416a-8819-59071c739f3a None admin] RBAC: Adding query filter params (name=service) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:35.139743 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef5ef34-d123-416a-8819-59071c739f3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:35.141260 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef5ef34-d123-416a-8819-59071c739f3a None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:35.150393 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 228/456] 10.222.0.39 () {56 vars in 1103 bytes} [Fri Oct 14 02:49:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:35.155258 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:35.155937 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef5ef34-d123-416a-8819-59071c739f3a None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:35.194919 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:35.227386 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c90d503-1a3c-4d84-8723-a097db53a2e3 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:35.263746 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84652357-cba4-45df-89a7-833b248caed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:35.264080 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef5ef34-d123-416a-8819-59071c739f3a None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:35.264353 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84652357-cba4-45df-89a7-833b248caed3 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:35.264632 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84652357-cba4-45df-89a7-833b248caed3 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:35.264900 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84652357-cba4-45df-89a7-833b248caed3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:35.304017 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7129231-de68-47f0-b397-7ed6db33f860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:35.304949 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7129231-de68-47f0-b397-7ed6db33f860 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:35.304949 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7129231-de68-47f0-b397-7ed6db33f860 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:35.305149 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7129231-de68-47f0-b397-7ed6db33f860 None admin] PATH_INFO: `/v3/users` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:35.305911 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7129231-de68-47f0-b397-7ed6db33f860 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:35.306221 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7129231-de68-47f0-b397-7ed6db33f860 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:35.307656 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7129231-de68-47f0-b397-7ed6db33f860 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:35.325427 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 229/457] 10.222.0.39 () {56 vars in 1137 bytes} [Fri Oct 14 02:49:35 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:35.326290 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7129231-de68-47f0-b397-7ed6db33f860 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:35.341029 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84652357-cba4-45df-89a7-833b248caed3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:35.342689 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84652357-cba4-45df-89a7-833b248caed3 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:35.384263 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:35.420145 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:35.428405 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7129231-de68-47f0-b397-7ed6db33f860 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:35.455948 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 229/458] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Oct 14 02:49:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 14 02:49:35.461449 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84652357-cba4-45df-89a7-833b248caed3 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:35.464818 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2134d25-a32e-47d4-b45e-8885510acd99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:35.465639 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2134d25-a32e-47d4-b45e-8885510acd99 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:35.465952 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2134d25-a32e-47d4-b45e-8885510acd99 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:35.466225 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2134d25-a32e-47d4-b45e-8885510acd99 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:35.474016 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2134d25-a32e-47d4-b45e-8885510acd99 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:35.475461 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2134d25-a32e-47d4-b45e-8885510acd99 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:35.487967 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-a2134d25-a32e-47d4-b45e-8885510acd99 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 14 02:49:35.487967 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:35.487967 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:35.487967 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:35.487967 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:35.487967 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:35.487967 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:35.487967 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:35.487967 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:35.487967 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:35.487967 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:35.487967 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:35.487967 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 14 02:49:35.487967 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 14 02:49:35.487967 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 14 02:49:35.487967 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 14 02:49:35.488925 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:35.488925 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:35.488925 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 14 02:49:35.488925 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:49:35.488925 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 14 02:49:35.488925 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:49:35.488925 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:35.488925 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:35.488925 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:35.488925 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:35.488925 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:35.488925 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:35.488925 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:35.488925 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:35.488925 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:35.488925 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:35.488925 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:35.488925 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:35.489860 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:35.489860 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:35.489860 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 14 02:49:35.489860 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 14 02:49:35.489860 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 14 02:49:35.489860 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 14 02:49:35.489860 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 14 02:49:35.489860 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 14 02:49:35.489860 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 14 02:49:35.489860 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:49:35.489860 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 230/459] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Oct 14 02:49:35 2022] GET /identity/v3/users/glance => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:35.489860 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2134d25-a32e-47d4-b45e-8885510acd99 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:35.571712 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84652357-cba4-45df-89a7-833b248caed3 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:35.598472 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2134d25-a32e-47d4-b45e-8885510acd99 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:35.610705 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b9f971-9ced-4b4a-b555-54ba3c138108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:35.611344 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b9f971-9ced-4b4a-b555-54ba3c138108 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:35.611629 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b9f971-9ced-4b4a-b555-54ba3c138108 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:35.611907 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b9f971-9ced-4b4a-b555-54ba3c138108 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:35.615376 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:35.623458 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b9f971-9ced-4b4a-b555-54ba3c138108 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:35.624908 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b9f971-9ced-4b4a-b555-54ba3c138108 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:35.634174 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaead72-9e42-49ea-9595-c90436cd153b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:35.634778 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaead72-9e42-49ea-9595-c90436cd153b None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:35.635059 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaead72-9e42-49ea-9595-c90436cd153b None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:35.635329 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaead72-9e42-49ea-9595-c90436cd153b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:35.640759 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-a9b9f971-9ced-4b4a-b555-54ba3c138108 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:35.640759 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:35.640759 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:35.640759 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:35.640759 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:35.640759 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:35.640759 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:35.640759 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:35.640759 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:35.640759 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:35.640759 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:35.640759 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:35.640759 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:49:35.640759 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:49:35.640759 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:49:35.640759 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:49:35.641653 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:35.641653 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:35.641653 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:35.641653 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:35.641653 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:35.641653 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:35.641653 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:35.641653 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:35.641653 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:35.641653 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:35.641653 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:35.641653 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:35.641653 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:35.641653 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:35.641653 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:35.641653 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:35.641653 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:49:35.642532 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:49:35.642532 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:35.642532 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:35.647067 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 230/460] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:49:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:35.647814 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b9f971-9ced-4b4a-b555-54ba3c138108 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:35.659517 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:35.708578 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaead72-9e42-49ea-9595-c90436cd153b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:35.710064 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaead72-9e42-49ea-9595-c90436cd153b None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:35.753774 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b9f971-9ced-4b4a-b555-54ba3c138108 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:35.790857 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5f781b-53cf-4794-b210-c0020032b287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:35.791602 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5f781b-53cf-4794-b210-c0020032b287 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:35.791883 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5f781b-53cf-4794-b210-c0020032b287 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:35.792164 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5f781b-53cf-4794-b210-c0020032b287 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:35.803741 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5f781b-53cf-4794-b210-c0020032b287 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:35.805236 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5f781b-53cf-4794-b210-c0020032b287 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:35.814161 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 231/461] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Oct 14 02:49:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 14 02:49:35.814161 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaead72-9e42-49ea-9595-c90436cd153b None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:35.824114 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-4d5f781b-53cf-4794-b210-c0020032b287 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:35.824114 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:35.824114 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:35.824114 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:35.824114 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:35.824114 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:35.824114 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:35.824114 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:35.824114 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:35.824114 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:35.824114 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:35.824114 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:35.824114 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:49:35.824114 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:49:35.824114 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:49:35.824114 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:49:35.825033 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:35.825033 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:35.825033 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:35.825033 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:35.825033 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:35.825033 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:35.825033 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:35.825033 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:35.825033 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:35.825033 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:35.825033 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:35.825033 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:35.825033 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:35.825033 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:35.825033 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:35.825033 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:35.825033 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:49:35.825900 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:49:35.825900 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:35.825900 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:35.825900 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 231/462] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:49:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:35.826542 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5f781b-53cf-4794-b210-c0020032b287 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:35.873750 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:35.876011 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:35.925582 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaead72-9e42-49ea-9595-c90436cd153b None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:35.933432 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5f781b-53cf-4794-b210-c0020032b287 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:35.963859 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a611a22b-1420-452a-8bd3-a38c05403001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:35.964898 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a611a22b-1420-452a-8bd3-a38c05403001 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:35.965314 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a611a22b-1420-452a-8bd3-a38c05403001 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:35.965598 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a611a22b-1420-452a-8bd3-a38c05403001 None admin] PATH_INFO: `/v3/users` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:35.966367 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a611a22b-1420-452a-8bd3-a38c05403001 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:35.966659 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a611a22b-1420-452a-8bd3-a38c05403001 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:35.968155 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a611a22b-1420-452a-8bd3-a38c05403001 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:35.968663 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8de5ad-7b34-4941-b257-be08949af55a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:35.969262 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8de5ad-7b34-4941-b257-be08949af55a None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:35.969542 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8de5ad-7b34-4941-b257-be08949af55a None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:35.969862 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8de5ad-7b34-4941-b257-be08949af55a None admin] PATH_INFO: `/v3/domains` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:35.970533 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8de5ad-7b34-4941-b257-be08949af55a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:35.970834 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8de5ad-7b34-4941-b257-be08949af55a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:35.972212 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8de5ad-7b34-4941-b257-be08949af55a None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:35.983786 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 232/463] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Oct 14 02:49:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:35.984567 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8de5ad-7b34-4941-b257-be08949af55a None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:35.992580 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 232/464] 10.222.0.39 () {56 vars in 1137 bytes} [Fri Oct 14 02:49:35 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:35.993386 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a611a22b-1420-452a-8bd3-a38c05403001 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:36.092170 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:36.095849 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8de5ad-7b34-4941-b257-be08949af55a None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:36.098899 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a611a22b-1420-452a-8bd3-a38c05403001 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:36.109671 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:36.133627 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984249d3-bbf5-43fe-aabf-9dc504516d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:36.133943 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658438b3-1c32-4118-ac2d-8ee155d6534f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:36.134287 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984249d3-bbf5-43fe-aabf-9dc504516d53 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:36.134546 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658438b3-1c32-4118-ac2d-8ee155d6534f None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:36.134645 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984249d3-bbf5-43fe-aabf-9dc504516d53 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:36.134839 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984249d3-bbf5-43fe-aabf-9dc504516d53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:36.135511 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658438b3-1c32-4118-ac2d-8ee155d6534f None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:36.135842 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658438b3-1c32-4118-ac2d-8ee155d6534f None admin] PATH_INFO: `/v3/domains` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:36.136480 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658438b3-1c32-4118-ac2d-8ee155d6534f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:36.136787 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658438b3-1c32-4118-ac2d-8ee155d6534f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:36.138185 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658438b3-1c32-4118-ac2d-8ee155d6534f None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:36.145856 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984249d3-bbf5-43fe-aabf-9dc504516d53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:36.147547 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984249d3-bbf5-43fe-aabf-9dc504516d53 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:36.150012 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 233/465] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Oct 14 02:49:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:36.150821 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658438b3-1c32-4118-ac2d-8ee155d6534f None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:36.170409 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-984249d3-bbf5-43fe-aabf-9dc504516d53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:36.170409 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:36.170409 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:36.170409 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:36.170409 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:36.170409 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:36.170409 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:36.170409 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:36.170409 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:36.170409 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:36.170409 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:36.170409 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:36.170409 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:49:36.170409 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:49:36.170409 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:49:36.170409 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:49:36.171538 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:36.171538 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:36.171538 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:36.171538 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:36.171538 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:36.171538 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:36.171538 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:36.171538 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:36.171538 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:36.171538 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:36.171538 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:36.171538 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:36.171538 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:36.171538 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:36.171538 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:36.171538 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:36.171538 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:49:36.172436 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:49:36.172436 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:36.172436 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:36.172436 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 233/466] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:49:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:36.172657 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984249d3-bbf5-43fe-aabf-9dc504516d53 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:36.254378 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658438b3-1c32-4118-ac2d-8ee155d6534f None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:36.280957 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984249d3-bbf5-43fe-aabf-9dc504516d53 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:36.287243 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814d3aa3-4d0c-482d-aefa-d068615cfe49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:36.287868 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814d3aa3-4d0c-482d-aefa-d068615cfe49 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:36.288152 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814d3aa3-4d0c-482d-aefa-d068615cfe49 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:36.288429 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814d3aa3-4d0c-482d-aefa-d068615cfe49 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:36.295764 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814d3aa3-4d0c-482d-aefa-d068615cfe49 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:36.297218 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814d3aa3-4d0c-482d-aefa-d068615cfe49 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:36.308675 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-814d3aa3-4d0c-482d-aefa-d068615cfe49 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 14 02:49:36.308675 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:36.308675 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:36.308675 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:36.308675 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:36.308675 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:36.308675 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:36.308675 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:36.308675 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:36.308675 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:36.308675 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:36.308675 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:36.308675 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 14 02:49:36.308675 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 14 02:49:36.308675 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 14 02:49:36.308675 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 14 02:49:36.309580 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:36.309580 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:36.309580 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 14 02:49:36.309580 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:49:36.309580 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 14 02:49:36.309580 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:49:36.309580 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:36.309580 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:36.309580 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:36.309580 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:36.309580 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:36.309580 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:36.309580 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:36.309580 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:36.309580 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:36.309580 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:36.309580 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:36.309580 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:36.310497 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:36.310497 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:36.310497 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 14 02:49:36.310497 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 14 02:49:36.310497 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 14 02:49:36.310497 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 14 02:49:36.310497 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 14 02:49:36.310497 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 14 02:49:36.310497 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 14 02:49:36.310497 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:49:36.310497 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 234/467] 10.222.0.39 () {56 vars in 1102 bytes} [Fri Oct 14 02:49:36 2022] GET /identity/v3/users/neutron => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:36.310497 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814d3aa3-4d0c-482d-aefa-d068615cfe49 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:36.322615 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90521267-3b2b-4b87-aa01-0bd6d62fa16c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:36.323404 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90521267-3b2b-4b87-aa01-0bd6d62fa16c None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:36.323845 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90521267-3b2b-4b87-aa01-0bd6d62fa16c None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:36.324385 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90521267-3b2b-4b87-aa01-0bd6d62fa16c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:36.342968 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:36.347494 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:36.397894 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90521267-3b2b-4b87-aa01-0bd6d62fa16c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:36.399334 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90521267-3b2b-4b87-aa01-0bd6d62fa16c None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:36.414817 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814d3aa3-4d0c-482d-aefa-d068615cfe49 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:36.451791 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca01eefc-8162-4eb7-a605-c756d6c076d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:36.452406 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca01eefc-8162-4eb7-a605-c756d6c076d7 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:36.452696 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca01eefc-8162-4eb7-a605-c756d6c076d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:36.453143 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca01eefc-8162-4eb7-a605-c756d6c076d7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:36.460794 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca01eefc-8162-4eb7-a605-c756d6c076d7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:36.462377 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca01eefc-8162-4eb7-a605-c756d6c076d7 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:36.474370 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-ca01eefc-8162-4eb7-a605-c756d6c076d7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:49:36.474370 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:36.474370 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:36.474370 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:36.474370 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:36.474370 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:36.474370 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:36.474370 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:36.474370 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:36.474370 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:36.474370 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:36.474370 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:36.474370 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 14 02:49:36.474370 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 14 02:49:36.474370 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 14 02:49:36.475396 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 14 02:49:36.475396 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:36.475396 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:36.475396 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:36.475396 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:36.475396 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:36.475396 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:36.475396 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:36.475396 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:36.475396 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:36.475396 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:36.475396 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:36.475396 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:36.475396 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:36.475396 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:36.475396 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:36.475396 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:36.476426 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 14 02:49:36.476426 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 14 02:49:36.476426 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 14 02:49:36.476426 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 14 02:49:36.476426 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 14 02:49:36.476426 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 14 02:49:36.476426 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:49:36.476426 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:49:36.480469 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 235/468] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Oct 14 02:49:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:36.481067 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca01eefc-8162-4eb7-a605-c756d6c076d7 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:36.513056 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 234/469] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Oct 14 02:49:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 14 02:49:36.513643 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90521267-3b2b-4b87-aa01-0bd6d62fa16c None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:36.565641 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:36.579824 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:36.589457 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca01eefc-8162-4eb7-a605-c756d6c076d7 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:36.613749 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90521267-3b2b-4b87-aa01-0bd6d62fa16c None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:36.624191 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9b6afc-cc61-400d-b7ae-08036af4cb98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:36.624797 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9b6afc-cc61-400d-b7ae-08036af4cb98 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:36.625081 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9b6afc-cc61-400d-b7ae-08036af4cb98 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:36.625359 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9b6afc-cc61-400d-b7ae-08036af4cb98 None admin] PATH_INFO: `/v3/domains` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:36.626071 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9b6afc-cc61-400d-b7ae-08036af4cb98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:36.626376 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9b6afc-cc61-400d-b7ae-08036af4cb98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:36.627756 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9b6afc-cc61-400d-b7ae-08036af4cb98 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:36.639710 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 236/470] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Oct 14 02:49:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:36.640549 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9b6afc-cc61-400d-b7ae-08036af4cb98 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:36.657867 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dfc168-e978-4b64-8a57-5f37f59dd7c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:36.658510 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dfc168-e978-4b64-8a57-5f37f59dd7c7 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:36.658793 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dfc168-e978-4b64-8a57-5f37f59dd7c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:36.659073 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dfc168-e978-4b64-8a57-5f37f59dd7c7 None admin] PATH_INFO: `/v3/users` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:36.659861 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dfc168-e978-4b64-8a57-5f37f59dd7c7 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:36.660175 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dfc168-e978-4b64-8a57-5f37f59dd7c7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:36.661639 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dfc168-e978-4b64-8a57-5f37f59dd7c7 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:36.685555 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 235/471] 10.222.0.39 () {56 vars in 1139 bytes} [Fri Oct 14 02:49:36 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:36.685555 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dfc168-e978-4b64-8a57-5f37f59dd7c7 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:36.743017 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9b6afc-cc61-400d-b7ae-08036af4cb98 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:36.779086 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ddc8a1-6ff4-4303-ada6-f901f8570b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:36.779613 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ddc8a1-6ff4-4303-ada6-f901f8570b8b None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:36.779901 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ddc8a1-6ff4-4303-ada6-f901f8570b8b None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:36.780181 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ddc8a1-6ff4-4303-ada6-f901f8570b8b None admin] PATH_INFO: `/v3/projects` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:36.780870 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ddc8a1-6ff4-4303-ada6-f901f8570b8b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:36.781180 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ddc8a1-6ff4-4303-ada6-f901f8570b8b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:36.782648 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ddc8a1-6ff4-4303-ada6-f901f8570b8b None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:36.789689 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dfc168-e978-4b64-8a57-5f37f59dd7c7 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:36.799590 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 237/472] 10.222.0.39 () {56 vars in 1148 bytes} [Fri Oct 14 02:49:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:36.799590 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ddc8a1-6ff4-4303-ada6-f901f8570b8b None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:36.804455 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:36.812696 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:36.828890 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e8d6cd-6ed2-4a28-b318-0e2ac031696d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:36.829521 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e8d6cd-6ed2-4a28-b318-0e2ac031696d None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:36.829833 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e8d6cd-6ed2-4a28-b318-0e2ac031696d None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:36.830118 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e8d6cd-6ed2-4a28-b318-0e2ac031696d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:36.840555 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e8d6cd-6ed2-4a28-b318-0e2ac031696d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:36.842040 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e8d6cd-6ed2-4a28-b318-0e2ac031696d None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:36.856997 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-f3e8d6cd-6ed2-4a28-b318-0e2ac031696d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:36.856997 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:36.856997 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:36.856997 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:36.856997 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:36.856997 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:36.856997 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:36.856997 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:36.856997 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:36.856997 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:36.856997 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:36.856997 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:36.856997 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:49:36.856997 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:49:36.856997 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:49:36.856997 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:49:36.857950 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:36.857950 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:36.857950 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:36.857950 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:36.857950 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:36.857950 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:36.857950 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:36.857950 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:36.857950 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:36.857950 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:36.857950 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:36.857950 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:36.857950 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:36.857950 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:36.857950 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:36.857950 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:36.857950 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:49:36.858778 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:49:36.858778 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:36.858778 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:36.858778 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 236/473] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:49:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:36.859421 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e8d6cd-6ed2-4a28-b318-0e2ac031696d None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:36.908049 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ddc8a1-6ff4-4303-ada6-f901f8570b8b None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:36.943233 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e86d85a-3aea-4bb1-965f-3defcee9a8ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:36.943864 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e86d85a-3aea-4bb1-965f-3defcee9a8ea None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:36.944150 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e86d85a-3aea-4bb1-965f-3defcee9a8ea None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:36.944425 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e86d85a-3aea-4bb1-965f-3defcee9a8ea None admin] PATH_INFO: `/v3/projects/service` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:36.951160 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e86d85a-3aea-4bb1-965f-3defcee9a8ea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:36.952619 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e86d85a-3aea-4bb1-965f-3defcee9a8ea None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:36.965280 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-3e86d85a-3aea-4bb1-965f-3defcee9a8ea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:49:36.965280 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:36.965280 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:36.965280 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:36.965280 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:36.965280 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:36.965280 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:36.965280 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:36.965280 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:36.965280 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:36.965280 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:36.965280 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:36.965280 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 14 02:49:36.965280 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 14 02:49:36.965280 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 14 02:49:36.966157 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 14 02:49:36.966157 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:36.966157 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:36.966157 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:36.966157 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:36.966157 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:36.966157 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:36.966157 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:36.966157 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:36.966157 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:36.966157 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:36.966157 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:36.966157 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:36.966157 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:36.966157 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:36.966157 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:36.966157 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:36.966992 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 14 02:49:36.966992 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 14 02:49:36.966992 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 14 02:49:36.966992 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 14 02:49:36.966992 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 14 02:49:36.966992 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 14 02:49:36.966992 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:49:36.966992 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:49:36.966992 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 238/474] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Oct 14 02:49:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:36.967443 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e86d85a-3aea-4bb1-965f-3defcee9a8ea None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:36.970036 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e8d6cd-6ed2-4a28-b318-0e2ac031696d None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:37.012867 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81094543-7fdf-4f88-93e5-674384faa43c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:37.013505 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81094543-7fdf-4f88-93e5-674384faa43c None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:37.013814 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81094543-7fdf-4f88-93e5-674384faa43c None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:37.014096 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81094543-7fdf-4f88-93e5-674384faa43c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:37.048550 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:37.049346 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:37.079290 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e86d85a-3aea-4bb1-965f-3defcee9a8ea None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:37.088956 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81094543-7fdf-4f88-93e5-674384faa43c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:37.090460 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81094543-7fdf-4f88-93e5-674384faa43c None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:37.114871 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eabbcd4-d065-4d22-8c81-1412739ffe86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:37.115483 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eabbcd4-d065-4d22-8c81-1412739ffe86 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:37.115766 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eabbcd4-d065-4d22-8c81-1412739ffe86 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:37.116034 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eabbcd4-d065-4d22-8c81-1412739ffe86 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:37.116762 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eabbcd4-d065-4d22-8c81-1412739ffe86 None admin] RBAC: Adding query filter params (role.id=9c7226aa19e742759967212b7291782b, scope.project.id=dac9f8cd285c44e889a621b8b7fe479e, user.id=a6b07bf0faca47b28b50990d1720853f) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:37.117071 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eabbcd4-d065-4d22-8c81-1412739ffe86 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:37.118560 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eabbcd4-d065-4d22-8c81-1412739ffe86 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:37.124678 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 239/475] 10.222.0.39 () {56 vars in 1374 bytes} [Fri Oct 14 02:49:36 2022] GET /identity/v3/role_assignments?user.id=a6b07bf0faca47b28b50990d1720853f&scope.project.id=dac9f8cd285c44e889a621b8b7fe479e&role.id=9c7226aa19e742759967212b7291782b => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:37.125664 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eabbcd4-d065-4d22-8c81-1412739ffe86 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:37.202626 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 237/476] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Oct 14 02:49:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 14 02:49:37.203534 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81094543-7fdf-4f88-93e5-674384faa43c None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:37.248316 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eabbcd4-d065-4d22-8c81-1412739ffe86 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:37.275834 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:37.285442 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af40c69e-6f95-48bc-817f-ffd8ec794081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:37.286662 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af40c69e-6f95-48bc-817f-ffd8ec794081 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:37.287330 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af40c69e-6f95-48bc-817f-ffd8ec794081 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:37.287330 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af40c69e-6f95-48bc-817f-ffd8ec794081 None admin] PATH_INFO: `/v3/domains` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:37.287844 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af40c69e-6f95-48bc-817f-ffd8ec794081 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:37.288143 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af40c69e-6f95-48bc-817f-ffd8ec794081 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:37.289657 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af40c69e-6f95-48bc-817f-ffd8ec794081 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:37.291475 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:37.302639 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 240/477] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Oct 14 02:49:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:37.307169 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8248b2c-b326-4e45-9c05-d4ff4f0aa785 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:37.307523 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8248b2c-b326-4e45-9c05-d4ff4f0aa785 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:37.307869 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8248b2c-b326-4e45-9c05-d4ff4f0aa785 None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:37.309191 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 241/478] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:49:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:49:37.309911 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8248b2c-b326-4e45-9c05-d4ff4f0aa785 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:37.323669 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81094543-7fdf-4f88-93e5-674384faa43c None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:37.363375 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc13d3d-a78c-4818-b1e0-7dcacf74895f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:37.364032 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc13d3d-a78c-4818-b1e0-7dcacf74895f None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:37.364321 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc13d3d-a78c-4818-b1e0-7dcacf74895f None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:37.364579 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc13d3d-a78c-4818-b1e0-7dcacf74895f None admin] PATH_INFO: `/v3/projects` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:37.365271 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc13d3d-a78c-4818-b1e0-7dcacf74895f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:37.365573 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc13d3d-a78c-4818-b1e0-7dcacf74895f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:37.367057 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc13d3d-a78c-4818-b1e0-7dcacf74895f None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:37.383643 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 238/479] 10.222.0.39 () {56 vars in 1148 bytes} [Fri Oct 14 02:49:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:37.383643 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc13d3d-a78c-4818-b1e0-7dcacf74895f None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:37.422810 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8248b2c-b326-4e45-9c05-d4ff4f0aa785 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:37.461116 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61705bc-f85d-4965-bc20-76812c41aa9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:37.461744 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61705bc-f85d-4965-bc20-76812c41aa9d None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:37.462050 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61705bc-f85d-4965-bc20-76812c41aa9d None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:37.462321 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61705bc-f85d-4965-bc20-76812c41aa9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:37.475987 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61705bc-f85d-4965-bc20-76812c41aa9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:37.477468 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61705bc-f85d-4965-bc20-76812c41aa9d None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:37.490367 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc13d3d-a78c-4818-b1e0-7dcacf74895f None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:37.492694 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-e61705bc-f85d-4965-bc20-76812c41aa9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:37.492694 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:37.492694 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:37.492694 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:37.492694 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:37.492694 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:37.492694 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:37.492694 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:37.492694 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:37.492694 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:37.492694 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:37.492694 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:37.492694 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:49:37.492694 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:49:37.492694 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:49:37.492694 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:49:37.493568 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:37.493568 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:37.493568 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:37.493568 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:37.493568 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:37.493568 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:37.493568 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:37.493568 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:37.493568 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:37.493568 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:37.493568 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:37.493568 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:37.493568 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:37.493568 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:37.493568 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:37.493568 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:37.493568 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:49:37.494461 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:49:37.494461 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:37.494461 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:49:37.494461 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 242/480] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:49:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:37.495017 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9d5ef5-cfd0-48bc-9ed5-00a0639008cf None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:37.495298 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9d5ef5-cfd0-48bc-9ed5-00a0639008cf None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:37.495563 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9d5ef5-cfd0-48bc-9ed5-00a0639008cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:37.513385 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-ef9d5ef5-cfd0-48bc-9ed5-00a0639008cf None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:49:37.514599 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-ef9d5ef5-cfd0-48bc-9ed5-00a0639008cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:49:37.516427 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:37.528565 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3186f74a-15ba-4a47-9096-c9a452a3f83d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:37.529362 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3186f74a-15ba-4a47-9096-c9a452a3f83d None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:37.529645 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3186f74a-15ba-4a47-9096-c9a452a3f83d None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:37.529956 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3186f74a-15ba-4a47-9096-c9a452a3f83d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:37.537900 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3186f74a-15ba-4a47-9096-c9a452a3f83d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:37.539348 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3186f74a-15ba-4a47-9096-c9a452a3f83d None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:37.550838 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-3186f74a-15ba-4a47-9096-c9a452a3f83d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:49:37.550838 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:37.550838 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:37.550838 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:37.550838 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:37.550838 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:37.550838 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:37.550838 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:37.550838 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:37.550838 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:37.550838 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:37.550838 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:37.550838 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 14 02:49:37.550838 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 14 02:49:37.550838 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 14 02:49:37.551707 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 14 02:49:37.551707 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:37.551707 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:37.551707 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:37.551707 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:37.551707 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:37.551707 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:37.551707 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:37.551707 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:37.551707 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:37.551707 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:37.551707 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:37.551707 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:37.551707 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:37.551707 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:37.551707 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:37.551707 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:37.552579 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 14 02:49:37.552579 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 14 02:49:37.552579 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 14 02:49:37.552579 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 14 02:49:37.552579 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 14 02:49:37.552579 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 14 02:49:37.552579 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:49:37.552579 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:37.552579 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 239/481] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Oct 14 02:49:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:37.553107 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3186f74a-15ba-4a47-9096-c9a452a3f83d None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:37.567252 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:37.593800 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 243/482] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:49:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:49:37.593800 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9d5ef5-cfd0-48bc-9ed5-00a0639008cf None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:37.686237 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3186f74a-15ba-4a47-9096-c9a452a3f83d None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:37.707440 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9d5ef5-cfd0-48bc-9ed5-00a0639008cf None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:37.730186 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61666b9a-815c-4ba7-ab5f-7abca1f6373e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:37.731280 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61666b9a-815c-4ba7-ab5f-7abca1f6373e None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:37.731701 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61666b9a-815c-4ba7-ab5f-7abca1f6373e None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:37.732201 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61666b9a-815c-4ba7-ab5f-7abca1f6373e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:37.733056 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61666b9a-815c-4ba7-ab5f-7abca1f6373e None admin] RBAC: Adding query filter params (role.id=9c7226aa19e742759967212b7291782b, scope.project.id=dac9f8cd285c44e889a621b8b7fe479e, user.id=ab599d98b0624a2e89090312bcab713f) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:37.733915 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61666b9a-815c-4ba7-ab5f-7abca1f6373e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:37.735742 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61666b9a-815c-4ba7-ab5f-7abca1f6373e None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:37.743995 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 240/483] 10.222.0.39 () {56 vars in 1374 bytes} [Fri Oct 14 02:49:37 2022] GET /identity/v3/role_assignments?user.id=ab599d98b0624a2e89090312bcab713f&scope.project.id=dac9f8cd285c44e889a621b8b7fe479e&role.id=9c7226aa19e742759967212b7291782b => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:37.745245 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61666b9a-815c-4ba7-ab5f-7abca1f6373e None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:37.750042 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fe9986-df50-4c7a-886b-c4760ca93991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:37.750954 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fe9986-df50-4c7a-886b-c4760ca93991 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:37.751387 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fe9986-df50-4c7a-886b-c4760ca93991 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:37.751831 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fe9986-df50-4c7a-886b-c4760ca93991 None admin] PATH_INFO: `/v3/domains` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:37.752653 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fe9986-df50-4c7a-886b-c4760ca93991 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:37.753118 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fe9986-df50-4c7a-886b-c4760ca93991 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:37.756002 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fe9986-df50-4c7a-886b-c4760ca93991 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:37.774249 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 244/484] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Oct 14 02:49:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:37.779138 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fe9986-df50-4c7a-886b-c4760ca93991 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:37.834875 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:37.872355 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:37.913226 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61666b9a-815c-4ba7-ab5f-7abca1f6373e None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:37.948153 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fe9986-df50-4c7a-886b-c4760ca93991 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:37.951698 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aa8929-b384-4cbd-9ec9-fca79b499155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:37.952634 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aa8929-b384-4cbd-9ec9-fca79b499155 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:37.953131 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aa8929-b384-4cbd-9ec9-fca79b499155 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:37.953609 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aa8929-b384-4cbd-9ec9-fca79b499155 None admin] PATH_INFO: `/v3/projects` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:37.954536 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8aa8929-b384-4cbd-9ec9-fca79b499155 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:37.955051 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8aa8929-b384-4cbd-9ec9-fca79b499155 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:37.956879 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8aa8929-b384-4cbd-9ec9-fca79b499155 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:37.972403 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 241/485] 10.222.0.39 () {56 vars in 1148 bytes} [Fri Oct 14 02:49:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:37.973607 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aa8929-b384-4cbd-9ec9-fca79b499155 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:37.993960 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11d8e7b-f4ab-453c-8d14-c2acd0d17a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:37.994593 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11d8e7b-f4ab-453c-8d14-c2acd0d17a53 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:37.994879 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11d8e7b-f4ab-453c-8d14-c2acd0d17a53 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:37.995154 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11d8e7b-f4ab-453c-8d14-c2acd0d17a53 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:38.006561 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11d8e7b-f4ab-453c-8d14-c2acd0d17a53 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:38.008919 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11d8e7b-f4ab-453c-8d14-c2acd0d17a53 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:38.022898 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-d11d8e7b-f4ab-453c-8d14-c2acd0d17a53 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:49:38.022898 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:38.022898 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:38.022898 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:38.022898 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:38.022898 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:38.022898 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:38.022898 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:38.022898 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:38.022898 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:38.022898 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:38.022898 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:38.022898 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 14 02:49:38.022898 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 14 02:49:38.022898 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 14 02:49:38.023735 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 14 02:49:38.023735 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:38.023735 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:38.023735 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:38.023735 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:38.023735 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:38.023735 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:38.023735 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:38.023735 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:38.023735 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:38.023735 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:38.023735 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:38.023735 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:38.023735 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:38.023735 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:38.023735 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:38.023735 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:38.024551 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 14 02:49:38.024551 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 14 02:49:38.024551 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 14 02:49:38.024551 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 14 02:49:38.024551 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 14 02:49:38.024551 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 14 02:49:38.024551 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:49:38.024551 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:49:38.025383 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 245/486] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Oct 14 02:49:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 02:49:38.025980 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11d8e7b-f4ab-453c-8d14-c2acd0d17a53 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:38.110782 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aa8929-b384-4cbd-9ec9-fca79b499155 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:38.120364 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:38.152618 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888755cd-63ad-4b03-bb77-fa4474c7b5fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:38.153427 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888755cd-63ad-4b03-bb77-fa4474c7b5fa None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:38.153716 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888755cd-63ad-4b03-bb77-fa4474c7b5fa None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:38.154023 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888755cd-63ad-4b03-bb77-fa4474c7b5fa None admin] PATH_INFO: `/v3/roles/service` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:38.154946 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:38.162226 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11d8e7b-f4ab-453c-8d14-c2acd0d17a53 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:38.165809 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888755cd-63ad-4b03-bb77-fa4474c7b5fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:49:38.170954 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-888755cd-63ad-4b03-bb77-fa4474c7b5fa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 14 02:49:38.170954 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:38.170954 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:38.170954 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:38.170954 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:38.170954 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:38.170954 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:38.170954 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:38.170954 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:38.170954 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:38.170954 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:38.170954 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:38.170954 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:49:38.170954 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:49:38.170954 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:49:38.170954 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:49:38.172028 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:49:38.172028 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:49:38.172028 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:49:38.172028 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:49:38.172028 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:38.172028 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:38.172028 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:38.172028 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:38.172028 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:38.172028 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:38.172028 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:38.172028 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:38.172028 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:38.172028 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:38.172028 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:38.172028 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:38.173096 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:38.173096 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:38.173096 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:38.173096 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:38.173096 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:49:38.173096 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:49:38.173096 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:49:38.173096 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:49:38.173096 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:49:38.173096 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:49:38.173096 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 14 02:49:38.173096 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:38.173096 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 242/487] 10.222.0.39 () {56 vars in 1102 bytes} [Fri Oct 14 02:49:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:38.178319 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888755cd-63ad-4b03-bb77-fa4474c7b5fa None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:38.210415 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fe3cf7-a5c2-4d4b-b49f-4f0e303f5802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:38.211030 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fe3cf7-a5c2-4d4b-b49f-4f0e303f5802 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:38.211726 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fe3cf7-a5c2-4d4b-b49f-4f0e303f5802 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:38.212033 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fe3cf7-a5c2-4d4b-b49f-4f0e303f5802 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:38.213036 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fe3cf7-a5c2-4d4b-b49f-4f0e303f5802 None admin] RBAC: Adding query filter params (role.id=9c7226aa19e742759967212b7291782b, scope.project.id=dac9f8cd285c44e889a621b8b7fe479e, user.id=6a3cd1fbe6df48f8aa90d4ea69905e9e) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:38.213361 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fe3cf7-a5c2-4d4b-b49f-4f0e303f5802 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:38.215509 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fe3cf7-a5c2-4d4b-b49f-4f0e303f5802 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:38.223504 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 246/488] 10.222.0.39 () {56 vars in 1374 bytes} [Fri Oct 14 02:49:38 2022] GET /identity/v3/role_assignments?user.id=6a3cd1fbe6df48f8aa90d4ea69905e9e&scope.project.id=dac9f8cd285c44e889a621b8b7fe479e&role.id=9c7226aa19e742759967212b7291782b => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:38.224422 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fe3cf7-a5c2-4d4b-b49f-4f0e303f5802 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:38.372610 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888755cd-63ad-4b03-bb77-fa4474c7b5fa None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:38.388779 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fe3cf7-a5c2-4d4b-b49f-4f0e303f5802 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:38.422873 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd37ed2d-ffd1-4230-a89f-891176461959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:38.423975 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd37ed2d-ffd1-4230-a89f-891176461959 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:38.424409 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd37ed2d-ffd1-4230-a89f-891176461959 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:38.424833 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd37ed2d-ffd1-4230-a89f-891176461959 None admin] PATH_INFO: `/v3/projects` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:38.425698 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd37ed2d-ffd1-4230-a89f-891176461959 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:38.426293 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd37ed2d-ffd1-4230-a89f-891176461959 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:38.428094 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c961c60-ce13-4ac3-b0d9-6749ed00f028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:38.428539 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd37ed2d-ffd1-4230-a89f-891176461959 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:38.428706 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c961c60-ce13-4ac3-b0d9-6749ed00f028 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:38.428991 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c961c60-ce13-4ac3-b0d9-6749ed00f028 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:38.429266 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c961c60-ce13-4ac3-b0d9-6749ed00f028 None admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:38.429959 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c961c60-ce13-4ac3-b0d9-6749ed00f028 None admin] RBAC: Adding query filter params (name=service) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:38.430265 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c961c60-ce13-4ac3-b0d9-6749ed00f028 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:38.431768 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c961c60-ce13-4ac3-b0d9-6749ed00f028 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:38.440558 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 247/489] 10.222.0.39 () {56 vars in 1103 bytes} [Fri Oct 14 02:49:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:38.445319 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 243/490] 10.222.0.39 () {56 vars in 1148 bytes} [Fri Oct 14 02:49:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:38.451158 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd37ed2d-ffd1-4230-a89f-891176461959 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:38.451299 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c961c60-ce13-4ac3-b0d9-6749ed00f028 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:38.480272 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:38.487038 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:38.567218 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c961c60-ce13-4ac3-b0d9-6749ed00f028 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:38.576889 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd37ed2d-ffd1-4230-a89f-891176461959 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:38.607414 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e16e822-2fb5-480f-a81e-e88f920a25f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:38.608403 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e16e822-2fb5-480f-a81e-e88f920a25f1 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:38.608833 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e16e822-2fb5-480f-a81e-e88f920a25f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:38.609250 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e16e822-2fb5-480f-a81e-e88f920a25f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:38.611227 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e16e822-2fb5-480f-a81e-e88f920a25f1 None admin] RBAC: Adding query filter params (role.id=9c7226aa19e742759967212b7291782b, scope.project.id=dac9f8cd285c44e889a621b8b7fe479e, user.id=b123c412ac714b06929a4bd86eca71e0) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:38.611696 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e16e822-2fb5-480f-a81e-e88f920a25f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:38.613517 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e16e822-2fb5-480f-a81e-e88f920a25f1 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:38.616524 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74a7287-0dcf-4f98-9416-e61c409e1d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:38.617872 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74a7287-0dcf-4f98-9416-e61c409e1d77 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:38.617872 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74a7287-0dcf-4f98-9416-e61c409e1d77 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:38.617872 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74a7287-0dcf-4f98-9416-e61c409e1d77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:38.625743 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 248/491] 10.222.0.39 () {56 vars in 1374 bytes} [Fri Oct 14 02:49:38 2022] GET /identity/v3/role_assignments?user.id=b123c412ac714b06929a4bd86eca71e0&scope.project.id=dac9f8cd285c44e889a621b8b7fe479e&role.id=9c7226aa19e742759967212b7291782b => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:38.630831 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74a7287-0dcf-4f98-9416-e61c409e1d77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:38.632419 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74a7287-0dcf-4f98-9416-e61c409e1d77 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:38.649259 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-d74a7287-0dcf-4f98-9416-e61c409e1d77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:38.649259 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:38.649259 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:38.649259 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:38.649259 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:38.649259 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:38.649259 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:38.649259 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:38.649259 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:38.649259 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:38.649259 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:38.649259 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:38.649259 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:49:38.649259 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:49:38.649259 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:49:38.649259 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:49:38.651191 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:38.651191 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:38.651191 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:38.651191 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:38.651191 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:38.651191 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:38.651191 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:38.651191 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:38.651191 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:38.651191 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:38.651191 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:38.651191 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:38.651191 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:38.651191 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:38.651191 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:38.651191 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:38.651191 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:49:38.652325 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:49:38.652325 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:38.652325 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:38.652325 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 244/492] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:49:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:38.659930 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e16e822-2fb5-480f-a81e-e88f920a25f1 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:38.759710 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:38.777174 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e16e822-2fb5-480f-a81e-e88f920a25f1 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:38.818449 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d07027-9fdc-493f-9bad-71ed38afc55b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:38.819081 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d07027-9fdc-493f-9bad-71ed38afc55b None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:38.819417 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d07027-9fdc-493f-9bad-71ed38afc55b None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:38.819696 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d07027-9fdc-493f-9bad-71ed38afc55b None admin] PATH_INFO: `/v3/domains` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:38.820353 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d07027-9fdc-493f-9bad-71ed38afc55b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:38.820661 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d07027-9fdc-493f-9bad-71ed38afc55b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:38.822080 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d07027-9fdc-493f-9bad-71ed38afc55b None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:38.839772 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 249/493] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Oct 14 02:49:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:38.842971 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74a7287-0dcf-4f98-9416-e61c409e1d77 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:38.936051 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:38.958202 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74a7287-0dcf-4f98-9416-e61c409e1d77 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:38.993627 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d11e57-1850-4118-ae0e-a8adf3600d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:38.994302 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d11e57-1850-4118-ae0e-a8adf3600d68 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:38.994586 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d11e57-1850-4118-ae0e-a8adf3600d68 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:38.994860 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d11e57-1850-4118-ae0e-a8adf3600d68 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:39.004746 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d11e57-1850-4118-ae0e-a8adf3600d68 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:39.004746 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d11e57-1850-4118-ae0e-a8adf3600d68 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:39.016560 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-11d11e57-1850-4118-ae0e-a8adf3600d68 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 14 02:49:39.016560 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:39.016560 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:39.016560 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:39.016560 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:39.016560 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:39.016560 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:39.016560 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:39.016560 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:39.016560 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:39.016560 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:39.016560 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:39.016560 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 14 02:49:39.016560 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 14 02:49:39.016560 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 14 02:49:39.016560 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 14 02:49:39.017472 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:39.017472 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:39.017472 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 14 02:49:39.017472 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:49:39.017472 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 14 02:49:39.017472 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:49:39.017472 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:39.017472 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:39.017472 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:39.017472 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:39.017472 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:39.017472 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:39.017472 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:39.017472 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:39.017472 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:39.017472 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:39.017472 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:39.017472 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:39.018909 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:39.018909 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:39.018909 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 14 02:49:39.018909 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 14 02:49:39.018909 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 14 02:49:39.018909 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 14 02:49:39.018909 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 14 02:49:39.018909 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 14 02:49:39.018909 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 14 02:49:39.018909 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:39.018909 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 245/494] 10.222.0.39 () {56 vars in 1096 bytes} [Fri Oct 14 02:49:38 2022] GET /identity/v3/users/swift => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:39.026147 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d07027-9fdc-493f-9bad-71ed38afc55b None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:39.155773 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d07027-9fdc-493f-9bad-71ed38afc55b None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:39.204887 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b051f81-f13a-4b0c-8cbd-22562542087e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:39.205886 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b051f81-f13a-4b0c-8cbd-22562542087e None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:39.206414 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b051f81-f13a-4b0c-8cbd-22562542087e None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:39.206893 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b051f81-f13a-4b0c-8cbd-22562542087e None admin] PATH_INFO: `/v3/users` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:39.208427 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b051f81-f13a-4b0c-8cbd-22562542087e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:39.208941 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b051f81-f13a-4b0c-8cbd-22562542087e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:39.210618 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b051f81-f13a-4b0c-8cbd-22562542087e None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:39.235675 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 250/495] 10.222.0.39 () {56 vars in 1135 bytes} [Fri Oct 14 02:49:39 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:49:39.241423 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d11e57-1850-4118-ae0e-a8adf3600d68 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:39.364210 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d11e57-1850-4118-ae0e-a8adf3600d68 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:39.399353 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092a2061-ccec-4c14-8300-c412c71319c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:39.399967 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092a2061-ccec-4c14-8300-c412c71319c6 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:39.400254 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092a2061-ccec-4c14-8300-c412c71319c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:39.400530 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092a2061-ccec-4c14-8300-c412c71319c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:39.409111 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:39.465095 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe8030-1eca-4f2b-a682-802c62ea7c02 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:39.466050 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe8030-1eca-4f2b-a682-802c62ea7c02 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:39.466552 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe8030-1eca-4f2b-a682-802c62ea7c02 None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:39.468193 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 251/496] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:49:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:49:39.472185 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092a2061-ccec-4c14-8300-c412c71319c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:39.472390 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092a2061-ccec-4c14-8300-c412c71319c6 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:39.474190 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d293eeda-2ab6-4dd6-9377-700df47137f1 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:39.474474 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d293eeda-2ab6-4dd6-9377-700df47137f1 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:39.474750 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d293eeda-2ab6-4dd6-9377-700df47137f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:39.498837 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:39.522761 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-d293eeda-2ab6-4dd6-9377-700df47137f1 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:49:39.523955 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-d293eeda-2ab6-4dd6-9377-700df47137f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:49:39.578936 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 246/497] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Oct 14 02:49:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 14 02:49:39.584201 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092a2061-ccec-4c14-8300-c412c71319c6 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:39.636765 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 252/498] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:49:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:49:39.663381 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:39.749254 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092a2061-ccec-4c14-8300-c412c71319c6 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:39.787223 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e6c3cb-3b0b-4c06-90c4-2035eafa8928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:39.787864 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e6c3cb-3b0b-4c06-90c4-2035eafa8928 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:39.788145 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e6c3cb-3b0b-4c06-90c4-2035eafa8928 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:39.788417 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e6c3cb-3b0b-4c06-90c4-2035eafa8928 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:39.798859 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e6c3cb-3b0b-4c06-90c4-2035eafa8928 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:39.800344 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e6c3cb-3b0b-4c06-90c4-2035eafa8928 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:39.815923 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-79e6c3cb-3b0b-4c06-90c4-2035eafa8928 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:39.815923 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:39.815923 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:39.815923 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:39.815923 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:39.815923 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:39.815923 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:39.815923 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:39.815923 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:39.815923 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:39.815923 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:39.815923 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:39.815923 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:49:39.815923 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:49:39.815923 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:49:39.815923 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:49:39.817462 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:39.817462 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:39.817462 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:39.817462 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:39.817462 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:39.817462 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:39.817462 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:39.817462 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:39.817462 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:39.817462 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:39.817462 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:39.817462 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:39.817462 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:39.817462 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:39.817462 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:39.817462 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:39.817462 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:49:39.819062 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:49:39.819062 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:39.819062 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:39.819062 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 247/499] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:49:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:39.823167 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d293eeda-2ab6-4dd6-9377-700df47137f1 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:39.937562 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e6c3cb-3b0b-4c06-90c4-2035eafa8928 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:39.976141 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d293eeda-2ab6-4dd6-9377-700df47137f1 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:40.061653 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eff063f-7c38-41e5-a17f-6abeb1c08be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:40.062644 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eff063f-7c38-41e5-a17f-6abeb1c08be0 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:40.063133 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eff063f-7c38-41e5-a17f-6abeb1c08be0 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:40.063609 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eff063f-7c38-41e5-a17f-6abeb1c08be0 None admin] PATH_INFO: `/v3/domains` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:40.064454 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eff063f-7c38-41e5-a17f-6abeb1c08be0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:40.065839 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eff063f-7c38-41e5-a17f-6abeb1c08be0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:40.067711 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eff063f-7c38-41e5-a17f-6abeb1c08be0 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:40.075381 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:40.088783 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 253/500] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Oct 14 02:49:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:40.097342 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eff063f-7c38-41e5-a17f-6abeb1c08be0 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:40.132687 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:40.150171 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e6c3cb-3b0b-4c06-90c4-2035eafa8928 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:40.227436 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da490a7-5c1a-4eb1-8900-880e2a7bc38e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:40.228441 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da490a7-5c1a-4eb1-8900-880e2a7bc38e None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:40.231732 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da490a7-5c1a-4eb1-8900-880e2a7bc38e None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:40.232221 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da490a7-5c1a-4eb1-8900-880e2a7bc38e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:40.253566 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da490a7-5c1a-4eb1-8900-880e2a7bc38e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:49:40.264105 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-4da490a7-5c1a-4eb1-8900-880e2a7bc38e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:49:40.264105 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:40.264105 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:40.264105 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:40.264105 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:40.264105 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:40.264105 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:40.264105 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:40.264105 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:40.264105 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:40.264105 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:40.264105 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:40.264105 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:49:40.264105 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:49:40.264105 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:49:40.264105 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:49:40.265033 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:49:40.265033 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:49:40.265033 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:49:40.265033 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:49:40.265033 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:40.265033 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:40.265033 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:40.265033 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:40.265033 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:40.265033 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:40.265033 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:40.265033 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:40.265033 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:40.265033 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:40.265033 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:40.265033 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:40.266372 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:40.266372 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:40.266372 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:40.266372 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:40.266372 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:49:40.266372 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:49:40.266372 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:49:40.266372 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:49:40.266372 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:49:40.266372 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:49:40.266372 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:49:40.266372 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:40.270372 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 248/501] 10.222.0.39 () {56 vars in 1096 bytes} [Fri Oct 14 02:49:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:40.277535 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da490a7-5c1a-4eb1-8900-880e2a7bc38e None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:40.309789 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eff063f-7c38-41e5-a17f-6abeb1c08be0 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:40.362637 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f311bf-c695-484c-8954-c539eead2005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:40.363271 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f311bf-c695-484c-8954-c539eead2005 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:40.363556 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f311bf-c695-484c-8954-c539eead2005 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:40.363861 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f311bf-c695-484c-8954-c539eead2005 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:40.373603 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f311bf-c695-484c-8954-c539eead2005 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:40.375454 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f311bf-c695-484c-8954-c539eead2005 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:40.390298 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-f4f311bf-c695-484c-8954-c539eead2005 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:49:40.390298 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:40.390298 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:40.390298 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:40.390298 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:40.390298 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:40.390298 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:40.390298 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:40.390298 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:40.390298 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:40.390298 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:40.390298 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:40.390298 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 14 02:49:40.390298 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 14 02:49:40.390298 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 14 02:49:40.391224 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 14 02:49:40.391224 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:40.391224 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:40.391224 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:40.391224 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:40.391224 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:40.391224 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:40.391224 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:40.391224 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:40.391224 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:40.391224 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:40.391224 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:40.391224 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:40.391224 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:40.391224 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:40.391224 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:40.391224 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:40.392106 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 14 02:49:40.392106 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 14 02:49:40.392106 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 14 02:49:40.392106 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 14 02:49:40.392106 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 14 02:49:40.392106 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 14 02:49:40.392106 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:49:40.392106 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:49:40.392106 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 254/502] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Oct 14 02:49:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 02:49:40.393147 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68adbfb5-0e86-4d3c-bc7a-719150a9d7d7 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:40.393352 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68adbfb5-0e86-4d3c-bc7a-719150a9d7d7 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:40.393638 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68adbfb5-0e86-4d3c-bc7a-719150a9d7d7 None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:40.397864 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 255/503] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:49:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:49:40.398894 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68adbfb5-0e86-4d3c-bc7a-719150a9d7d7 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:40.426718 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da490a7-5c1a-4eb1-8900-880e2a7bc38e None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:40.462863 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49bd88c-805a-4b67-a469-d6a9666fa434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:40.463469 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49bd88c-805a-4b67-a469-d6a9666fa434 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:40.463922 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49bd88c-805a-4b67-a469-d6a9666fa434 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:40.464197 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49bd88c-805a-4b67-a469-d6a9666fa434 None admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:40.464973 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49bd88c-805a-4b67-a469-d6a9666fa434 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:40.465279 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49bd88c-805a-4b67-a469-d6a9666fa434 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:40.466677 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49bd88c-805a-4b67-a469-d6a9666fa434 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:40.476708 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 249/504] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Oct 14 02:49:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:40.477263 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:40.479968 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc10bd76-ca92-4380-9e32-d37239ff1782 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:40.480459 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc10bd76-ca92-4380-9e32-d37239ff1782 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:40.480952 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc10bd76-ca92-4380-9e32-d37239ff1782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:40.499472 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-dc10bd76-ca92-4380-9e32-d37239ff1782 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:49:40.500959 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-dc10bd76-ca92-4380-9e32-d37239ff1782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:49:40.509442 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:40.536096 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68adbfb5-0e86-4d3c-bc7a-719150a9d7d7 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:40.573446 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 250/505] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:49:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:49:40.574614 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc10bd76-ca92-4380-9e32-d37239ff1782 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:40.586575 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bba40e-2901-44da-8b25-6faadd421189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:40.587472 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bba40e-2901-44da-8b25-6faadd421189 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:40.587946 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bba40e-2901-44da-8b25-6faadd421189 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:40.588474 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bba40e-2901-44da-8b25-6faadd421189 None admin] PATH_INFO: `/v3/projects` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:40.589455 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bba40e-2901-44da-8b25-6faadd421189 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:40.591605 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bba40e-2901-44da-8b25-6faadd421189 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:40.593492 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bba40e-2901-44da-8b25-6faadd421189 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:40.618588 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 256/506] 10.222.0.39 () {56 vars in 1148 bytes} [Fri Oct 14 02:49:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:40.624745 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bba40e-2901-44da-8b25-6faadd421189 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:40.756164 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc10bd76-ca92-4380-9e32-d37239ff1782 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:40.798698 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bba40e-2901-44da-8b25-6faadd421189 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:40.814423 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:40.839545 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c970bd61-0033-46f7-a3b2-6382ba698959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:40.840630 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c970bd61-0033-46f7-a3b2-6382ba698959 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:40.841454 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c970bd61-0033-46f7-a3b2-6382ba698959 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:40.842423 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c970bd61-0033-46f7-a3b2-6382ba698959 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:40.846543 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afc3d45-450b-4250-9b32-15c26bc8d083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:40.847591 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afc3d45-450b-4250-9b32-15c26bc8d083 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:40.849211 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afc3d45-450b-4250-9b32-15c26bc8d083 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:40.849804 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afc3d45-450b-4250-9b32-15c26bc8d083 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:40.850810 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afc3d45-450b-4250-9b32-15c26bc8d083 None admin] RBAC: Adding query filter params (role.id=9c7226aa19e742759967212b7291782b, scope.project.id=dac9f8cd285c44e889a621b8b7fe479e, user.id=5c5e07da64804243bc18862072b027a1) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:40.851330 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afc3d45-450b-4250-9b32-15c26bc8d083 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:40.852952 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afc3d45-450b-4250-9b32-15c26bc8d083 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:40.858950 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c970bd61-0033-46f7-a3b2-6382ba698959 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:40.860712 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c970bd61-0033-46f7-a3b2-6382ba698959 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:40.862072 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 257/507] 10.222.0.39 () {56 vars in 1374 bytes} [Fri Oct 14 02:49:40 2022] GET /identity/v3/role_assignments?user.id=5c5e07da64804243bc18862072b027a1&scope.project.id=dac9f8cd285c44e889a621b8b7fe479e&role.id=9c7226aa19e742759967212b7291782b => generated 608 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:40.866676 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:40.890696 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-c970bd61-0033-46f7-a3b2-6382ba698959 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:40.890696 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:40.890696 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:40.890696 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:40.890696 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:40.890696 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:40.890696 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:40.890696 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:40.890696 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:40.890696 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:40.890696 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:40.890696 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:40.890696 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:49:40.890696 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:49:40.890696 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:49:40.890696 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:49:40.894848 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:40.894848 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:40.894848 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:40.894848 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:40.894848 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:40.894848 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:40.894848 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:40.894848 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:40.894848 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:40.894848 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:40.894848 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:40.894848 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:40.894848 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:40.894848 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:40.894848 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:40.894848 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:40.894848 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:49:40.895738 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:49:40.895738 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:40.895738 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:40.897715 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 251/508] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:49:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:40.903530 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afc3d45-450b-4250-9b32-15c26bc8d083 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:40.927139 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c970bd61-0033-46f7-a3b2-6382ba698959 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:41.090229 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afc3d45-450b-4250-9b32-15c26bc8d083 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:41.099679 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c970bd61-0033-46f7-a3b2-6382ba698959 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:41.138200 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10d1c25-b97d-47e7-bb8d-09a453bf343d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:41.138826 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10d1c25-b97d-47e7-bb8d-09a453bf343d None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:41.139283 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10d1c25-b97d-47e7-bb8d-09a453bf343d None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:41.139665 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10d1c25-b97d-47e7-bb8d-09a453bf343d None admin] PATH_INFO: `/v3/domains` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:41.140094 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10d1c25-b97d-47e7-bb8d-09a453bf343d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:41.140404 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10d1c25-b97d-47e7-bb8d-09a453bf343d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:41.142225 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10d1c25-b97d-47e7-bb8d-09a453bf343d None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:41.142730 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34594ce4-e228-4187-ac37-f90c07d7cc79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:41.143375 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34594ce4-e228-4187-ac37-f90c07d7cc79 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:41.143652 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34594ce4-e228-4187-ac37-f90c07d7cc79 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:41.143920 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34594ce4-e228-4187-ac37-f90c07d7cc79 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:41.144502 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34594ce4-e228-4187-ac37-f90c07d7cc79 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:41.146439 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34594ce4-e228-4187-ac37-f90c07d7cc79 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:41.158232 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-34594ce4-e228-4187-ac37-f90c07d7cc79 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 14 02:49:41.158232 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:41.158232 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 14 02:49:41.158232 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 14 02:49:41.158232 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 14 02:49:41.158232 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 14 02:49:41.158232 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 14 02:49:41.158232 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:49:41.158232 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 14 02:49:41.158232 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:41.158232 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 02:49:41.158232 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:41.158232 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:41.158232 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:41.158232 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:41.158232 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:41.158232 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:41.159508 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:41.159508 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:41.159508 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:41.159508 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:41.159508 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:41.159508 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:41.159508 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 14 02:49:41.159508 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 14 02:49:41.159508 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 14 02:49:41.159508 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 14 02:49:41.159508 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:41.159508 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:41.159508 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:41.159508 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:41.159508 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:41.159508 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:41.161193 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:41.161193 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:41.161193 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:41.161193 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:41.161193 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:41.161193 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:41.161193 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:41.161193 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:41.161193 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:41.161193 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:41.161193 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 14 02:49:41.161193 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:49:41.161193 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 14 02:49:41.161193 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:41.162038 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 258/509] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Oct 14 02:49:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:41.162913 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546f7f3b-dc5e-4317-9e05-a28bfb68022d None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:41.163201 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546f7f3b-dc5e-4317-9e05-a28bfb68022d None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:41.163477 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546f7f3b-dc5e-4317-9e05-a28bfb68022d None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:41.164127 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 252/510] 10.222.0.39 () {56 vars in 1129 bytes} [Fri Oct 14 02:49:40 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 02:49:41.164715 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 259/511] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:49:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:49:41.165284 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34594ce4-e228-4187-ac37-f90c07d7cc79 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:41.170380 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2355465-62ee-41db-a330-c50d285086f5 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:41.170659 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2355465-62ee-41db-a330-c50d285086f5 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:41.170928 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2355465-62ee-41db-a330-c50d285086f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:41.188884 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-f2355465-62ee-41db-a330-c50d285086f5 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:49:41.190160 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-f2355465-62ee-41db-a330-c50d285086f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:49:41.224190 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:41.238223 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:41.270365 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 260/512] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:49:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 14 02:49:41.271249 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2355465-62ee-41db-a330-c50d285086f5 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:41.289093 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34594ce4-e228-4187-ac37-f90c07d7cc79 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:41.354590 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe5a662-178e-4e33-80ee-881287335d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:41.355343 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe5a662-178e-4e33-80ee-881287335d09 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:41.355627 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe5a662-178e-4e33-80ee-881287335d09 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:41.355900 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe5a662-178e-4e33-80ee-881287335d09 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:41.371621 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe5a662-178e-4e33-80ee-881287335d09 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:41.373309 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe5a662-178e-4e33-80ee-881287335d09 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:41.397857 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-efe5a662-178e-4e33-80ee-881287335d09 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 14 02:49:41.397857 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:41.397857 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:41.397857 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:41.397857 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:41.397857 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:41.397857 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:41.397857 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:41.397857 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:41.397857 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:41.397857 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:41.397857 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:41.397857 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 14 02:49:41.397857 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 14 02:49:41.397857 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 14 02:49:41.397857 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 14 02:49:41.398892 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:41.398892 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:41.398892 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 14 02:49:41.398892 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:49:41.398892 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 14 02:49:41.398892 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:49:41.398892 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:41.398892 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:41.398892 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:41.398892 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:41.398892 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:41.398892 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:41.398892 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:41.398892 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:41.398892 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:41.398892 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:41.398892 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:41.398892 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:41.399839 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:41.399839 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:41.399839 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 14 02:49:41.399839 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 14 02:49:41.399839 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 14 02:49:41.399839 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 14 02:49:41.399839 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 14 02:49:41.399839 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 14 02:49:41.399839 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 14 02:49:41.399839 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:41.399839 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 253/513] 10.222.0.39 () {56 vars in 1093 bytes} [Fri Oct 14 02:49:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:41.406053 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe5a662-178e-4e33-80ee-881287335d09 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:41.468651 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2355465-62ee-41db-a330-c50d285086f5 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:41.505242 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b5cc4b-fad4-4fb9-9e26-727578212b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:41.506684 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b5cc4b-fad4-4fb9-9e26-727578212b31 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:41.507175 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b5cc4b-fad4-4fb9-9e26-727578212b31 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:41.507652 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b5cc4b-fad4-4fb9-9e26-727578212b31 None admin] PATH_INFO: `/v3/services` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:41.508484 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b5cc4b-fad4-4fb9-9e26-727578212b31 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:41.508992 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b5cc4b-fad4-4fb9-9e26-727578212b31 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:41.510619 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b5cc4b-fad4-4fb9-9e26-727578212b31 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:41.515824 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 261/514] 10.222.0.39 () {56 vars in 1124 bytes} [Fri Oct 14 02:49:41 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:41.522086 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b5cc4b-fad4-4fb9-9e26-727578212b31 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:41.543777 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe5a662-178e-4e33-80ee-881287335d09 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:41.554423 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:41.563288 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:41.581499 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0351323-8401-44f1-9b66-630b8f67f62f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:41.583009 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0351323-8401-44f1-9b66-630b8f67f62f None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:41.583436 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0351323-8401-44f1-9b66-630b8f67f62f None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:41.583887 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0351323-8401-44f1-9b66-630b8f67f62f None admin] PATH_INFO: `/v3/users` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:41.584781 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0351323-8401-44f1-9b66-630b8f67f62f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:41.585279 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0351323-8401-44f1-9b66-630b8f67f62f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:41.586941 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0351323-8401-44f1-9b66-630b8f67f62f None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:41.606733 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 254/515] 10.222.0.39 () {56 vars in 1133 bytes} [Fri Oct 14 02:49:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:49:41.608211 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea95e2b-3ef8-4f53-879b-6eec5d6d614f None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:41.608690 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea95e2b-3ef8-4f53-879b-6eec5d6d614f None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:41.609139 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea95e2b-3ef8-4f53-879b-6eec5d6d614f None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:41.610784 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 255/516] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:49:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:49:41.611699 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea95e2b-3ef8-4f53-879b-6eec5d6d614f None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:41.626916 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b5cc4b-fad4-4fb9-9e26-727578212b31 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:41.663925 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69927aa6-0920-4ad7-98d9-cfc4a28b90e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:41.664495 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69927aa6-0920-4ad7-98d9-cfc4a28b90e7 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:41.664763 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69927aa6-0920-4ad7-98d9-cfc4a28b90e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:41.665035 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69927aa6-0920-4ad7-98d9-cfc4a28b90e7 None admin] PATH_INFO: `/v3/services` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:41.665662 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69927aa6-0920-4ad7-98d9-cfc4a28b90e7 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:41.665995 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69927aa6-0920-4ad7-98d9-cfc4a28b90e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:41.667381 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69927aa6-0920-4ad7-98d9-cfc4a28b90e7 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:41.673348 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 262/517] 10.222.0.39 () {56 vars in 1124 bytes} [Fri Oct 14 02:49:41 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:49:41.683445 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d4460-2483-4efb-990f-bf11044723c1 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:41.683808 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d4460-2483-4efb-990f-bf11044723c1 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:41.684002 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d4460-2483-4efb-990f-bf11044723c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:41.704285 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-139d4460-2483-4efb-990f-bf11044723c1 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:49:41.705635 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-139d4460-2483-4efb-990f-bf11044723c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:49:41.728082 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea95e2b-3ef8-4f53-879b-6eec5d6d614f None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:41.767166 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cdc2c0-b846-4ab4-bc0e-01f36db9d971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:41.768170 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cdc2c0-b846-4ab4-bc0e-01f36db9d971 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:41.768676 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cdc2c0-b846-4ab4-bc0e-01f36db9d971 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:41.769145 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cdc2c0-b846-4ab4-bc0e-01f36db9d971 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:41.780956 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cdc2c0-b846-4ab4-bc0e-01f36db9d971 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:41.782829 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cdc2c0-b846-4ab4-bc0e-01f36db9d971 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:41.798341 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 263/518] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:49:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:49:41.799276 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-54cdc2c0-b846-4ab4-bc0e-01f36db9d971 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:41.799276 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:41.799276 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:41.799276 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:41.799276 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:41.799276 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:41.799276 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:41.799276 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:41.799276 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:41.799276 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:41.799276 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:41.799276 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:41.799276 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:49:41.799276 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:49:41.799276 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:49:41.799276 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:49:41.800186 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:41.800186 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:41.800186 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:41.800186 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:41.800186 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:41.800186 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:41.800186 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:41.800186 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:41.800186 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:41.800186 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:41.800186 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:41.800186 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:41.800186 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:41.800186 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:41.800186 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:41.800186 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:41.800186 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:49:41.801638 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:49:41.801638 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:41.801638 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:41.802240 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139d4460-2483-4efb-990f-bf11044723c1 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:41.803742 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 256/519] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:49:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:41.816403 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cdc2c0-b846-4ab4-bc0e-01f36db9d971 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:41.832737 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:41.870266 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:41.921058 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139d4460-2483-4efb-990f-bf11044723c1 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:41.948752 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cdc2c0-b846-4ab4-bc0e-01f36db9d971 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:41.982596 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40533e5-bea6-44ce-af6a-895fb5e1cc69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:41.983296 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40533e5-bea6-44ce-af6a-895fb5e1cc69 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:41.983707 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40533e5-bea6-44ce-af6a-895fb5e1cc69 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:41.983978 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40533e5-bea6-44ce-af6a-895fb5e1cc69 None admin] PATH_INFO: `/v3/domains` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:41.984676 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40533e5-bea6-44ce-af6a-895fb5e1cc69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:41.985166 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40533e5-bea6-44ce-af6a-895fb5e1cc69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:41.986525 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40533e5-bea6-44ce-af6a-895fb5e1cc69 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:41.999069 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 257/520] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Oct 14 02:49:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:41.999895 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a815f7-944c-4255-9d80-e3959ac14780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:42.000704 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a815f7-944c-4255-9d80-e3959ac14780 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:42.001114 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a815f7-944c-4255-9d80-e3959ac14780 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:42.001517 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a815f7-944c-4255-9d80-e3959ac14780 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:42.005204 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40533e5-bea6-44ce-af6a-895fb5e1cc69 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:42.030659 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a815f7-944c-4255-9d80-e3959ac14780 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:42.032342 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a815f7-944c-4255-9d80-e3959ac14780 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:42.065226 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-88a815f7-944c-4255-9d80-e3959ac14780 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:42.065226 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:42.065226 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:42.065226 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:42.065226 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:42.065226 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:42.065226 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:42.065226 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:42.065226 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:42.065226 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:42.065226 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:42.065226 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:42.065226 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:49:42.065226 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:49:42.065226 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:49:42.065226 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:49:42.066239 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:42.066239 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:42.066239 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:42.066239 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:42.066239 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:42.066239 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:42.066239 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:42.066239 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:42.066239 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:42.066239 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:42.066239 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:42.066239 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:42.066239 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:42.066239 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:42.066239 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:42.066239 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:42.066239 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:49:42.067712 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:49:42.067712 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:42.067712 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:49:42.069004 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 264/521] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:49:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:42.074470 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a815f7-944c-4255-9d80-e3959ac14780 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:42.122234 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:42.142966 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40533e5-bea6-44ce-af6a-895fb5e1cc69 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:42.202801 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660c6de1-a73e-45be-b4a5-a4d7fd8a97bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:42.203405 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660c6de1-a73e-45be-b4a5-a4d7fd8a97bd None admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:42.203686 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660c6de1-a73e-45be-b4a5-a4d7fd8a97bd None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:42.204010 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660c6de1-a73e-45be-b4a5-a4d7fd8a97bd None admin] PATH_INFO: `/v3/users` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:42.204671 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660c6de1-a73e-45be-b4a5-a4d7fd8a97bd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:42.206275 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660c6de1-a73e-45be-b4a5-a4d7fd8a97bd None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:42.216256 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:42.239620 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a815f7-944c-4255-9d80-e3959ac14780 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:42.273214 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eef2690-320b-4d66-8a49-0d356c4809bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:42.273838 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eef2690-320b-4d66-8a49-0d356c4809bf None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:42.274172 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eef2690-320b-4d66-8a49-0d356c4809bf None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:42.274446 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eef2690-320b-4d66-8a49-0d356c4809bf None admin] PATH_INFO: `/v3/domains` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:42.275131 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eef2690-320b-4d66-8a49-0d356c4809bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:42.275438 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eef2690-320b-4d66-8a49-0d356c4809bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:42.276810 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eef2690-320b-4d66-8a49-0d356c4809bf None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:42.295881 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 265/522] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Oct 14 02:49:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:42.295881 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eef2690-320b-4d66-8a49-0d356c4809bf None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:42.336575 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 258/523] 10.222.0.39 () {60 vars in 1132 bytes} [Fri Oct 14 02:49:42 2022] POST /identity/v3/users => generated 254 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:49:42.336575 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660c6de1-a73e-45be-b4a5-a4d7fd8a97bd None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:42.471730 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eef2690-320b-4d66-8a49-0d356c4809bf None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:42.518982 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd5478f-b00d-4847-956e-9067fe70c14d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:42.519618 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd5478f-b00d-4847-956e-9067fe70c14d None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:42.521104 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd5478f-b00d-4847-956e-9067fe70c14d None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:42.521104 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd5478f-b00d-4847-956e-9067fe70c14d None admin] PATH_INFO: `/v3/services/network` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:42.521480 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd5478f-b00d-4847-956e-9067fe70c14d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:42.523872 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd5478f-b00d-4847-956e-9067fe70c14d None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:42.549237 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660c6de1-a73e-45be-b4a5-a4d7fd8a97bd None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:42.550851 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-cfd5478f-b00d-4847-956e-9067fe70c14d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 14 02:49:42.550851 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:42.550851 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 14 02:49:42.550851 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 14 02:49:42.550851 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 14 02:49:42.550851 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 14 02:49:42.550851 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 14 02:49:42.550851 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:49:42.550851 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 14 02:49:42.550851 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:49:42.550851 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 02:49:42.550851 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:49:42.550851 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:42.550851 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:42.550851 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:42.550851 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:42.550851 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:42.551879 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:42.551879 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:42.551879 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:42.551879 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:42.551879 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:42.551879 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:42.551879 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 14 02:49:42.551879 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 14 02:49:42.551879 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 14 02:49:42.551879 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 14 02:49:42.551879 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:42.551879 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:42.551879 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:42.551879 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:42.551879 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:42.551879 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:42.552739 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:42.552739 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:42.552739 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:42.552739 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:42.552739 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:42.552739 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:42.552739 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:42.552739 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:42.552739 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:42.552739 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:42.552739 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 14 02:49:42.552739 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:49:42.552739 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 14 02:49:42.552739 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:49:42.553480 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 266/524] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Oct 14 02:49:42 2022] GET /identity/v3/services/network => generated 88 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:42.563705 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd5478f-b00d-4847-956e-9067fe70c14d None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:42.572661 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:42.633645 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45a9f36-e18c-43d6-b874-468675f37a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:42.634349 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45a9f36-e18c-43d6-b874-468675f37a21 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:42.634634 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45a9f36-e18c-43d6-b874-468675f37a21 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:42.634909 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45a9f36-e18c-43d6-b874-468675f37a21 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:42.646317 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45a9f36-e18c-43d6-b874-468675f37a21 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:42.647801 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45a9f36-e18c-43d6-b874-468675f37a21 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:42.663475 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-b45a9f36-e18c-43d6-b874-468675f37a21 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:49:42.663475 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:42.663475 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:42.663475 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:42.663475 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:42.663475 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:42.663475 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:42.663475 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:42.663475 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:42.663475 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:42.663475 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:42.663475 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:42.663475 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 14 02:49:42.663475 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 14 02:49:42.663475 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 14 02:49:42.664430 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 14 02:49:42.664430 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:42.664430 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:42.664430 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:42.664430 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:42.664430 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:42.664430 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:42.664430 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:42.664430 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:42.664430 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:42.664430 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:42.664430 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:42.664430 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:42.664430 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:42.664430 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:42.664430 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:42.664430 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:42.665332 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 14 02:49:42.665332 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 14 02:49:42.665332 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 14 02:49:42.665332 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 14 02:49:42.665332 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 14 02:49:42.665332 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 14 02:49:42.665332 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:49:42.665332 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:42.665332 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 259/525] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Oct 14 02:49:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:42.670253 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45a9f36-e18c-43d6-b874-468675f37a21 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:42.675469 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:42.714235 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd5478f-b00d-4847-956e-9067fe70c14d None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:42.779461 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28c4282-1297-43cc-9020-bba042eaed98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:42.779988 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28c4282-1297-43cc-9020-bba042eaed98 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:42.780264 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28c4282-1297-43cc-9020-bba042eaed98 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:42.780535 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28c4282-1297-43cc-9020-bba042eaed98 None admin] PATH_INFO: `/v3/services` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:42.781168 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28c4282-1297-43cc-9020-bba042eaed98 None admin] RBAC: Adding query filter params (name=network) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:42.781469 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28c4282-1297-43cc-9020-bba042eaed98 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:42.782919 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28c4282-1297-43cc-9020-bba042eaed98 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:42.799366 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45a9f36-e18c-43d6-b874-468675f37a21 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:42.799950 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 267/526] 10.222.0.39 () {56 vars in 1112 bytes} [Fri Oct 14 02:49:42 2022] GET /identity/v3/services?name=network => generated 124 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:49:42.810728 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28c4282-1297-43cc-9020-bba042eaed98 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:42.841114 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7deae65-795f-48a7-86e5-fed996af2360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:42.842151 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7deae65-795f-48a7-86e5-fed996af2360 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:42.842631 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7deae65-795f-48a7-86e5-fed996af2360 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:42.843099 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7deae65-795f-48a7-86e5-fed996af2360 None admin] PATH_INFO: `/v3/projects` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:42.844068 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7deae65-795f-48a7-86e5-fed996af2360 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:42.844757 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7deae65-795f-48a7-86e5-fed996af2360 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:42.846486 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7deae65-795f-48a7-86e5-fed996af2360 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:42.861664 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 260/527] 10.222.0.39 () {56 vars in 1148 bytes} [Fri Oct 14 02:49:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:42.867796 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7deae65-795f-48a7-86e5-fed996af2360 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:42.938979 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:42.978776 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7deae65-795f-48a7-86e5-fed996af2360 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:42.985026 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:43.022988 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2157c778-c73a-4e15-853c-b6fc8b36b502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:43.024046 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2157c778-c73a-4e15-853c-b6fc8b36b502 None admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:43.024565 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2157c778-c73a-4e15-853c-b6fc8b36b502 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:43.025029 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2157c778-c73a-4e15-853c-b6fc8b36b502 None admin] PATH_INFO: `/v3/projects/dac9f8cd285c44e889a621b8b7fe479e/users/a6b07bf0faca47b28b50990d1720853f/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:43.027200 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2157c778-c73a-4e15-853c-b6fc8b36b502 None admin] RBAC: Authorizing `identity:create_grant(project_id=dac9f8cd285c44e889a621b8b7fe479e, user_id=a6b07bf0faca47b28b50990d1720853f, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:43.028872 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2157c778-c73a-4e15-853c-b6fc8b36b502 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:43.056629 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28c4282-1297-43cc-9020-bba042eaed98 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:43.095056 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc48c16-cc55-402e-b41b-6f3acb277478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:43.095657 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc48c16-cc55-402e-b41b-6f3acb277478 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:43.095988 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc48c16-cc55-402e-b41b-6f3acb277478 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:43.096369 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc48c16-cc55-402e-b41b-6f3acb277478 None admin] PATH_INFO: `/v3/services` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:43.097095 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc48c16-cc55-402e-b41b-6f3acb277478 None admin] RBAC: Adding query filter params (type=network) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:43.097398 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc48c16-cc55-402e-b41b-6f3acb277478 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:43.098876 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc48c16-cc55-402e-b41b-6f3acb277478 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:43.104618 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 268/528] 10.222.0.39 () {56 vars in 1112 bytes} [Fri Oct 14 02:49:42 2022] GET /identity/v3/services?type=network => generated 124 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:43.129907 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 261/529] 10.222.0.39 () {58 vars in 1439 bytes} [Fri Oct 14 02:49:42 2022] PUT /identity/v3/projects/dac9f8cd285c44e889a621b8b7fe479e/users/a6b07bf0faca47b28b50990d1720853f/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 02:49:43.392072 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2e51c0-0d04-42ea-ba7c-d357f8e8109a None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:43.392445 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2e51c0-0d04-42ea-ba7c-d357f8e8109a None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:43.392624 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2e51c0-0d04-42ea-ba7c-d357f8e8109a None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:43.393997 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 269/530] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:49:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:49:43.399681 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d527b030-a626-4cb7-89bc-c2e19c9d56e9 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:43.399974 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d527b030-a626-4cb7-89bc-c2e19c9d56e9 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:43.400255 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d527b030-a626-4cb7-89bc-c2e19c9d56e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:43.418637 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-d527b030-a626-4cb7-89bc-c2e19c9d56e9 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:49:43.419851 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-d527b030-a626-4cb7-89bc-c2e19c9d56e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:49:43.493872 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 262/531] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:49:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:49:43.823298 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2e51c0-0d04-42ea-ba7c-d357f8e8109a None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:43.943398 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2e51c0-0d04-42ea-ba7c-d357f8e8109a None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:43.985636 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:43.993499 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae163a7e-fc95-44dc-94e8-695d0c22c79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:43.994157 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae163a7e-fc95-44dc-94e8-695d0c22c79c None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:43.994437 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae163a7e-fc95-44dc-94e8-695d0c22c79c None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:43.994708 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae163a7e-fc95-44dc-94e8-695d0c22c79c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:44.009528 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae163a7e-fc95-44dc-94e8-695d0c22c79c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:49:44.014422 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-ae163a7e-fc95-44dc-94e8-695d0c22c79c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:49:44.014422 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:44.014422 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:44.014422 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:44.014422 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:44.014422 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:44.014422 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:44.014422 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:44.014422 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:44.014422 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:44.014422 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:44.014422 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:44.014422 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:49:44.014422 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:49:44.014422 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:49:44.014422 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:49:44.015329 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:49:44.015329 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:49:44.015329 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:49:44.015329 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:49:44.015329 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:44.015329 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:44.015329 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:44.015329 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:44.015329 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:44.015329 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:44.015329 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:44.015329 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:44.015329 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:44.015329 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:44.015329 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:44.015329 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:44.016182 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:44.016182 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:44.016182 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:44.016182 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:44.016182 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:49:44.016182 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:49:44.016182 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:49:44.016182 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:49:44.016182 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:49:44.016182 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:49:44.016182 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:49:44.016182 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:49:44.016182 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 270/532] 10.222.0.39 () {56 vars in 1096 bytes} [Fri Oct 14 02:49:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:44.021701 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d527b030-a626-4cb7-89bc-c2e19c9d56e9 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:44.104885 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:44.156793 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d527b030-a626-4cb7-89bc-c2e19c9d56e9 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:44.183915 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfac8b0-1c51-4ef2-80e2-7a5e66e0f5e9 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:44.184221 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfac8b0-1c51-4ef2-80e2-7a5e66e0f5e9 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:44.184464 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfac8b0-1c51-4ef2-80e2-7a5e66e0f5e9 None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:44.185831 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 271/533] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:49:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 14 02:49:44.193471 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed0791f-ff64-4c6f-8bd2-f25e15e84c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:44.194144 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed0791f-ff64-4c6f-8bd2-f25e15e84c00 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:44.194437 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed0791f-ff64-4c6f-8bd2-f25e15e84c00 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:44.194719 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed0791f-ff64-4c6f-8bd2-f25e15e84c00 None admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:44.195390 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed0791f-ff64-4c6f-8bd2-f25e15e84c00 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:44.195698 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed0791f-ff64-4c6f-8bd2-f25e15e84c00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:44.197079 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed0791f-ff64-4c6f-8bd2-f25e15e84c00 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:44.199114 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a2afe2-5e83-4425-91e1-6a08c024ddb9 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:44.199409 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a2afe2-5e83-4425-91e1-6a08c024ddb9 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:44.199694 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a2afe2-5e83-4425-91e1-6a08c024ddb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:44.206408 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 263/534] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Oct 14 02:49:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:44.214734 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed0791f-ff64-4c6f-8bd2-f25e15e84c00 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:44.246979 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-d3a2afe2-5e83-4425-91e1-6a08c024ddb9 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:49:44.248211 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-d3a2afe2-5e83-4425-91e1-6a08c024ddb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:49:44.344818 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed0791f-ff64-4c6f-8bd2-f25e15e84c00 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:44.382922 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:44.391043 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 272/535] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:49:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 14 02:49:44.412259 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a59061a-e5b7-4c1a-8bf8-aefb68c84391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:44.412889 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a59061a-e5b7-4c1a-8bf8-aefb68c84391 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:44.413166 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a59061a-e5b7-4c1a-8bf8-aefb68c84391 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:44.413438 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a59061a-e5b7-4c1a-8bf8-aefb68c84391 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:44.429886 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a59061a-e5b7-4c1a-8bf8-aefb68c84391 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:44.431387 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a59061a-e5b7-4c1a-8bf8-aefb68c84391 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:44.448590 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-2a59061a-e5b7-4c1a-8bf8-aefb68c84391 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:44.448590 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:44.448590 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:44.448590 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:44.448590 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:44.448590 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:44.448590 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:44.448590 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:44.448590 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:44.448590 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:44.448590 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:44.448590 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:44.448590 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:49:44.448590 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:49:44.448590 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:49:44.448590 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:49:44.449516 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:44.449516 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:44.449516 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:44.449516 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:44.449516 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:44.449516 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:44.449516 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:44.449516 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:44.449516 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:44.449516 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:44.449516 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:44.449516 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:44.449516 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:44.449516 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:44.449516 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:44.449516 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:44.449516 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:49:44.451470 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:49:44.451470 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:44.451470 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:44.451470 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 264/536] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:49:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:44.455675 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a2afe2-5e83-4425-91e1-6a08c024ddb9 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:44.565665 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a2afe2-5e83-4425-91e1-6a08c024ddb9 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:44.586200 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:44.601011 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740837b9-4ebf-412a-afd4-bd6474cf79ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:44.605913 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740837b9-4ebf-412a-afd4-bd6474cf79ec None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:44.605913 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740837b9-4ebf-412a-afd4-bd6474cf79ec None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:44.605913 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740837b9-4ebf-412a-afd4-bd6474cf79ec None admin] PATH_INFO: `/v3/domains` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:44.605913 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740837b9-4ebf-412a-afd4-bd6474cf79ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:44.605913 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740837b9-4ebf-412a-afd4-bd6474cf79ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:44.605913 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740837b9-4ebf-412a-afd4-bd6474cf79ec None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:44.617202 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 273/537] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Oct 14 02:49:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:44.622766 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a59061a-e5b7-4c1a-8bf8-aefb68c84391 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:44.717428 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740837b9-4ebf-412a-afd4-bd6474cf79ec None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:44.741743 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a59061a-e5b7-4c1a-8bf8-aefb68c84391 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:44.805383 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd80505f-7500-4c93-8f77-8425887ed8b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:44.806060 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd80505f-7500-4c93-8f77-8425887ed8b3 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:44.806352 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd80505f-7500-4c93-8f77-8425887ed8b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:44.806632 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd80505f-7500-4c93-8f77-8425887ed8b3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:44.822837 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd80505f-7500-4c93-8f77-8425887ed8b3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:44.824598 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd80505f-7500-4c93-8f77-8425887ed8b3 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:44.847214 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-fd80505f-7500-4c93-8f77-8425887ed8b3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 14 02:49:44.847214 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:44.847214 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:44.847214 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:44.847214 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:44.847214 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:44.847214 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:44.847214 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:44.847214 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:44.847214 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:44.847214 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:44.847214 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:44.847214 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 14 02:49:44.847214 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 14 02:49:44.847214 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 14 02:49:44.847214 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 14 02:49:44.848826 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:44.848826 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:44.848826 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 14 02:49:44.848826 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:49:44.848826 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 14 02:49:44.848826 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:49:44.848826 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:44.848826 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:44.848826 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:44.848826 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:44.848826 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:44.848826 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:44.848826 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:44.848826 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:44.848826 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:44.848826 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:44.848826 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:44.848826 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:44.849856 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:44.849856 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:44.849856 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 14 02:49:44.849856 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 14 02:49:44.849856 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 14 02:49:44.849856 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 14 02:49:44.849856 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 14 02:49:44.849856 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 14 02:49:44.849856 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 14 02:49:44.849856 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:44.852323 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 265/538] 10.222.0.39 () {56 vars in 1096 bytes} [Fri Oct 14 02:49:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:44.858620 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740837b9-4ebf-412a-afd4-bd6474cf79ec None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:44.860125 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd80505f-7500-4c93-8f77-8425887ed8b3 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:44.909418 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:44.953592 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75e0152-6ba7-410d-bd1b-0f05021f0109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:44.954343 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75e0152-6ba7-410d-bd1b-0f05021f0109 None admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:44.954570 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75e0152-6ba7-410d-bd1b-0f05021f0109 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:44.954845 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75e0152-6ba7-410d-bd1b-0f05021f0109 None admin] PATH_INFO: `/v3/services` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:44.955501 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75e0152-6ba7-410d-bd1b-0f05021f0109 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:44.956898 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75e0152-6ba7-410d-bd1b-0f05021f0109 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:44.989286 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd80505f-7500-4c93-8f77-8425887ed8b3 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:45.025313 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d1518a-5170-4464-ac77-dfc26160f064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:45.026340 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d1518a-5170-4464-ac77-dfc26160f064 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:45.026862 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d1518a-5170-4464-ac77-dfc26160f064 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:45.027328 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d1518a-5170-4464-ac77-dfc26160f064 None admin] PATH_INFO: `/v3/users` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:45.028298 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d1518a-5170-4464-ac77-dfc26160f064 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:45.028798 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d1518a-5170-4464-ac77-dfc26160f064 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:45.030545 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d1518a-5170-4464-ac77-dfc26160f064 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:45.049573 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 266/539] 10.222.0.39 () {56 vars in 1135 bytes} [Fri Oct 14 02:49:44 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:45.055900 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d1518a-5170-4464-ac77-dfc26160f064 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:45.091868 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 274/540] 10.222.0.39 () {60 vars in 1141 bytes} [Fri Oct 14 02:49:44 2022] POST /identity/v3/services => generated 251 bytes in 375 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:49:45.177805 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:45.203360 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d1518a-5170-4464-ac77-dfc26160f064 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:45.261319 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a390f234-ebf3-4400-bddd-357267662ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:45.261915 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a390f234-ebf3-4400-bddd-357267662ffe None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:45.262258 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a390f234-ebf3-4400-bddd-357267662ffe None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:45.262540 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a390f234-ebf3-4400-bddd-357267662ffe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:45.276293 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a390f234-ebf3-4400-bddd-357267662ffe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:45.276601 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a390f234-ebf3-4400-bddd-357267662ffe None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:45.283147 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28303b59-2391-4131-9a04-a2d2d4667b02 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:45.283439 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28303b59-2391-4131-9a04-a2d2d4667b02 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:45.283720 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28303b59-2391-4131-9a04-a2d2d4667b02 None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:45.285378 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 275/541] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:49:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:49:45.290863 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d869559-1c4c-4591-a5f0-f9626feaf210 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:45.291152 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d869559-1c4c-4591-a5f0-f9626feaf210 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:45.291434 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d869559-1c4c-4591-a5f0-f9626feaf210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:45.293608 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-a390f234-ebf3-4400-bddd-357267662ffe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:45.293608 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:45.293608 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:45.293608 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:45.293608 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:45.293608 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:45.293608 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:45.293608 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:45.293608 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:45.293608 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:45.293608 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:45.293608 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:45.293608 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:49:45.293608 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:49:45.293608 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:49:45.293608 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:49:45.294623 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:45.294623 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:45.294623 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:45.294623 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:45.294623 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:45.294623 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:45.294623 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:45.294623 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:45.294623 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:45.294623 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:45.294623 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:45.294623 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:45.294623 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:45.294623 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:45.294623 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:45.294623 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:45.294623 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:49:45.295528 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:49:45.295528 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:45.295528 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:45.297849 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 267/542] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:49:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:45.303304 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a390f234-ebf3-4400-bddd-357267662ffe None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:45.316151 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-4d869559-1c4c-4591-a5f0-f9626feaf210 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:49:45.317944 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-4d869559-1c4c-4591-a5f0-f9626feaf210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:49:45.357667 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:45.394567 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 276/543] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:49:45 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:49:45.415151 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a390f234-ebf3-4400-bddd-357267662ffe None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:45.449354 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b48467-48d8-462a-8dc3-250e8021fa4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:45.449997 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b48467-48d8-462a-8dc3-250e8021fa4e None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:45.450278 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b48467-48d8-462a-8dc3-250e8021fa4e None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:45.450549 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b48467-48d8-462a-8dc3-250e8021fa4e None admin] PATH_INFO: `/v3/domains` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:45.451188 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b48467-48d8-462a-8dc3-250e8021fa4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:45.451492 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b48467-48d8-462a-8dc3-250e8021fa4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:45.453217 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b48467-48d8-462a-8dc3-250e8021fa4e None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:45.454678 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:45.466263 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 268/544] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Oct 14 02:49:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:49:45.471904 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d869559-1c4c-4591-a5f0-f9626feaf210 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:45.580371 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d869559-1c4c-4591-a5f0-f9626feaf210 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:45.620887 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6af095-bd03-4fa7-946b-af02e13214ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:45.621873 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6af095-bd03-4fa7-946b-af02e13214ab None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:45.622327 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6af095-bd03-4fa7-946b-af02e13214ab None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:45.622771 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6af095-bd03-4fa7-946b-af02e13214ab None admin] PATH_INFO: `/v3/projects/service` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:45.630975 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6af095-bd03-4fa7-946b-af02e13214ab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:45.632661 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6af095-bd03-4fa7-946b-af02e13214ab None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:45.644046 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-4a6af095-bd03-4fa7-946b-af02e13214ab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:49:45.644046 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:45.644046 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:45.644046 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:45.644046 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:45.644046 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:45.644046 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:45.644046 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:45.644046 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:45.644046 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:45.644046 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:45.644046 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:45.644046 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 14 02:49:45.644046 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 14 02:49:45.644046 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 14 02:49:45.644948 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 14 02:49:45.644948 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:45.644948 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:45.644948 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:45.644948 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:45.644948 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:45.644948 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:45.644948 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:45.644948 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:45.644948 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:45.644948 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:45.644948 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:45.644948 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:45.644948 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:45.644948 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:45.644948 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:45.644948 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:45.645869 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 14 02:49:45.645869 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 14 02:49:45.645869 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 14 02:49:45.645869 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 14 02:49:45.645869 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 14 02:49:45.645869 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 14 02:49:45.645869 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:49:45.645869 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:49:45.647803 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 277/545] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Oct 14 02:49:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:45.653960 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b48467-48d8-462a-8dc3-250e8021fa4e None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:45.687288 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6af095-bd03-4fa7-946b-af02e13214ab None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:45.739563 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:45.763596 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b48467-48d8-462a-8dc3-250e8021fa4e None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:45.803502 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcf3f38-8854-46e9-9cbd-c7c417fc73d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:45.804691 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcf3f38-8854-46e9-9cbd-c7c417fc73d5 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:45.805259 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcf3f38-8854-46e9-9cbd-c7c417fc73d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:45.805796 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcf3f38-8854-46e9-9cbd-c7c417fc73d5 None admin] PATH_INFO: `/v3/projects` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:45.806641 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcf3f38-8854-46e9-9cbd-c7c417fc73d5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:45.807092 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcf3f38-8854-46e9-9cbd-c7c417fc73d5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:45.808738 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcf3f38-8854-46e9-9cbd-c7c417fc73d5 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:45.823330 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 269/546] 10.222.0.39 () {56 vars in 1148 bytes} [Fri Oct 14 02:49:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:45.829353 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d951fc5-4a3d-4625-9d74-de0589ee2823 None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:45.829504 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6af095-bd03-4fa7-946b-af02e13214ab None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:45.830183 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d951fc5-4a3d-4625-9d74-de0589ee2823 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:45.830519 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d951fc5-4a3d-4625-9d74-de0589ee2823 None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:45.831954 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 270/547] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:49:45 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:49:45.834300 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d951fc5-4a3d-4625-9d74-de0589ee2823 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:45.872786 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826ba397-29fb-48ef-a201-52975b25a29c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:45.873493 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826ba397-29fb-48ef-a201-52975b25a29c None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:45.873752 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826ba397-29fb-48ef-a201-52975b25a29c None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:45.874121 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826ba397-29fb-48ef-a201-52975b25a29c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:45.886027 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826ba397-29fb-48ef-a201-52975b25a29c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:49:45.890681 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-826ba397-29fb-48ef-a201-52975b25a29c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 14 02:49:45.890681 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:45.890681 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:45.890681 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:45.890681 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:45.890681 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:45.890681 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:45.890681 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:45.890681 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:45.890681 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:45.890681 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:45.890681 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:45.890681 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:49:45.890681 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:49:45.890681 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:49:45.890681 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:49:45.891592 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:49:45.891592 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:49:45.891592 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:49:45.891592 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:49:45.891592 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:45.891592 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:45.891592 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:45.891592 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:45.891592 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:45.891592 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:45.891592 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:45.891592 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:45.891592 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:45.891592 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:45.891592 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:45.891592 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:45.892403 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:45.892403 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:45.892403 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:45.892403 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:45.892403 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:49:45.892403 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:49:45.892403 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:49:45.892403 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:49:45.892403 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:49:45.892403 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:49:45.892403 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 14 02:49:45.892403 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:49:45.892403 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 278/548] 10.222.0.39 () {56 vars in 1102 bytes} [Fri Oct 14 02:49:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:45.894920 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cde0a7-f54d-4b70-b927-5ccb0673a133 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:45.895214 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cde0a7-f54d-4b70-b927-5ccb0673a133 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:45.895495 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cde0a7-f54d-4b70-b927-5ccb0673a133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:45.918014 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-d6cde0a7-f54d-4b70-b927-5ccb0673a133 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:49:45.919798 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-d6cde0a7-f54d-4b70-b927-5ccb0673a133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:49:45.928059 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:45.965362 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d951fc5-4a3d-4625-9d74-de0589ee2823 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:46.009782 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428484a2-a89e-4de1-8665-f19574caf9b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:46.009782 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428484a2-a89e-4de1-8665-f19574caf9b7 None admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:46.009782 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428484a2-a89e-4de1-8665-f19574caf9b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:46.009782 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428484a2-a89e-4de1-8665-f19574caf9b7 None admin] PATH_INFO: `/v3/projects/dac9f8cd285c44e889a621b8b7fe479e/users/5c5e07da64804243bc18862072b027a1/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:46.010340 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428484a2-a89e-4de1-8665-f19574caf9b7 None admin] RBAC: Authorizing `identity:create_grant(project_id=dac9f8cd285c44e889a621b8b7fe479e, user_id=5c5e07da64804243bc18862072b027a1, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:46.010747 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 279/549] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:49:45 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:49:46.012019 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428484a2-a89e-4de1-8665-f19574caf9b7 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:46.012536 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cde0a7-f54d-4b70-b927-5ccb0673a133 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:46.036904 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:46.083406 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 271/550] 10.222.0.39 () {58 vars in 1439 bytes} [Fri Oct 14 02:49:45 2022] PUT /identity/v3/projects/dac9f8cd285c44e889a621b8b7fe479e/users/5c5e07da64804243bc18862072b027a1/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:49:46.085070 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d17a2-39e6-46df-a942-436f6fa0a774 None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:46.085571 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d17a2-39e6-46df-a942-436f6fa0a774 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:46.086093 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d17a2-39e6-46df-a942-436f6fa0a774 None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:46.092473 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 272/551] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:49:46 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:49:46.094430 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19a0240-409b-4e29-b2ab-1cda9fb52049 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:46.094916 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19a0240-409b-4e29-b2ab-1cda9fb52049 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:46.096308 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19a0240-409b-4e29-b2ab-1cda9fb52049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:46.141550 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-d19a0240-409b-4e29-b2ab-1cda9fb52049 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:49:46.146021 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-d19a0240-409b-4e29-b2ab-1cda9fb52049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:49:46.165457 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cde0a7-f54d-4b70-b927-5ccb0673a133 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:46.210396 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b4626e-e9d5-4e08-9fca-af8d5a73c0eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:46.211414 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b4626e-e9d5-4e08-9fca-af8d5a73c0eb None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:46.211905 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b4626e-e9d5-4e08-9fca-af8d5a73c0eb None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:46.212355 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b4626e-e9d5-4e08-9fca-af8d5a73c0eb None admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:46.213267 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b4626e-e9d5-4e08-9fca-af8d5a73c0eb None admin] RBAC: Adding query filter params (name=service) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:46.213889 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b4626e-e9d5-4e08-9fca-af8d5a73c0eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:46.215481 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b4626e-e9d5-4e08-9fca-af8d5a73c0eb None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:46.225925 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 280/552] 10.222.0.39 () {56 vars in 1103 bytes} [Fri Oct 14 02:49:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:46.239561 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b4626e-e9d5-4e08-9fca-af8d5a73c0eb None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:46.271245 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 273/553] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:49:46 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 14 02:49:46.283042 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19a0240-409b-4e29-b2ab-1cda9fb52049 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:46.360981 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:46.379164 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:46.400907 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b4626e-e9d5-4e08-9fca-af8d5a73c0eb None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:46.422452 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19a0240-409b-4e29-b2ab-1cda9fb52049 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:46.450468 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591bcabc-5c98-4f19-be78-54ddbc82ba8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:46.451407 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591bcabc-5c98-4f19-be78-54ddbc82ba8e None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:46.451827 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591bcabc-5c98-4f19-be78-54ddbc82ba8e None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:46.452305 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591bcabc-5c98-4f19-be78-54ddbc82ba8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:46.458095 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cede245c-50cb-4f74-8336-e6d1818c3931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:46.458844 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cede245c-50cb-4f74-8336-e6d1818c3931 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:46.459222 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cede245c-50cb-4f74-8336-e6d1818c3931 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:46.459590 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cede245c-50cb-4f74-8336-e6d1818c3931 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:46.469365 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591bcabc-5c98-4f19-be78-54ddbc82ba8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:46.471105 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591bcabc-5c98-4f19-be78-54ddbc82ba8e None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:46.471498 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cede245c-50cb-4f74-8336-e6d1818c3931 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:49:46.476987 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-cede245c-50cb-4f74-8336-e6d1818c3931 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:49:46.476987 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:46.476987 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:46.476987 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:46.476987 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:46.476987 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:46.476987 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:46.476987 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:46.476987 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:46.476987 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:46.476987 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:46.476987 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:46.476987 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:49:46.476987 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:49:46.476987 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:49:46.476987 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:49:46.478136 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:49:46.478136 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:49:46.478136 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:49:46.478136 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:49:46.478136 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:46.478136 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:46.478136 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:46.478136 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:46.478136 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:46.478136 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:46.478136 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:46.478136 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:46.478136 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:46.478136 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:46.478136 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:46.478136 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:46.479192 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:46.479192 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:46.479192 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:46.479192 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:46.479192 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:49:46.479192 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:49:46.479192 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:49:46.479192 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:49:46.479192 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:49:46.479192 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:49:46.479192 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:49:46.479192 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:46.481286 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 274/554] 10.222.0.39 () {56 vars in 1096 bytes} [Fri Oct 14 02:49:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:46.487026 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cede245c-50cb-4f74-8336-e6d1818c3931 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:46.496455 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-591bcabc-5c98-4f19-be78-54ddbc82ba8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:46.496455 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:46.496455 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:46.496455 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:46.496455 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:46.496455 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:46.496455 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:46.496455 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:46.496455 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:46.496455 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:46.496455 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:46.496455 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:46.496455 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:49:46.496455 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:49:46.496455 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:49:46.496455 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:49:46.497587 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:46.497587 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:46.497587 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:46.497587 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:46.497587 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:46.497587 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:46.497587 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:46.497587 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:46.497587 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:46.497587 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:46.497587 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:46.497587 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:46.497587 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:46.497587 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:46.497587 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:46.497587 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:46.497587 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:49:46.500546 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:49:46.500546 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:46.500546 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:49:46.502390 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 281/555] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:49:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:46.510760 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591bcabc-5c98-4f19-be78-54ddbc82ba8e None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:46.590843 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cede245c-50cb-4f74-8336-e6d1818c3931 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:46.598621 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:46.626673 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77adc711-526e-4d8a-b2c2-d9c5fba3b9c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:46.627445 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77adc711-526e-4d8a-b2c2-d9c5fba3b9c5 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:46.627816 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77adc711-526e-4d8a-b2c2-d9c5fba3b9c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:46.628265 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77adc711-526e-4d8a-b2c2-d9c5fba3b9c5 None admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:46.629513 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77adc711-526e-4d8a-b2c2-d9c5fba3b9c5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:46.629943 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77adc711-526e-4d8a-b2c2-d9c5fba3b9c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:46.631399 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77adc711-526e-4d8a-b2c2-d9c5fba3b9c5 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:46.646073 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 275/556] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Oct 14 02:49:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:49:46.647087 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77adc711-526e-4d8a-b2c2-d9c5fba3b9c5 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:46.699931 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591bcabc-5c98-4f19-be78-54ddbc82ba8e None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:46.744684 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118f9db2-ab74-4b91-91be-961122c49639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:46.745878 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f9db2-ab74-4b91-91be-961122c49639 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:46.746253 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f9db2-ab74-4b91-91be-961122c49639 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:46.746644 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f9db2-ab74-4b91-91be-961122c49639 None admin] PATH_INFO: `/v3/domains` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:46.747386 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118f9db2-ab74-4b91-91be-961122c49639 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:46.747787 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118f9db2-ab74-4b91-91be-961122c49639 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:46.749845 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118f9db2-ab74-4b91-91be-961122c49639 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:46.751916 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:46.762902 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77adc711-526e-4d8a-b2c2-d9c5fba3b9c5 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:46.766997 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 282/557] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Oct 14 02:49:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:46.773275 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118f9db2-ab74-4b91-91be-961122c49639 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:46.807200 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63e8b32-c8d4-4ee2-bf33-5f763191ed1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:46.807871 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63e8b32-c8d4-4ee2-bf33-5f763191ed1f None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:46.808222 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63e8b32-c8d4-4ee2-bf33-5f763191ed1f None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:46.808506 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63e8b32-c8d4-4ee2-bf33-5f763191ed1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:46.822055 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63e8b32-c8d4-4ee2-bf33-5f763191ed1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:46.823575 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63e8b32-c8d4-4ee2-bf33-5f763191ed1f None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:46.841068 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-b63e8b32-c8d4-4ee2-bf33-5f763191ed1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:46.841068 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:46.841068 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:46.841068 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:46.841068 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:46.841068 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:46.841068 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:46.841068 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:46.841068 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:46.841068 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:46.841068 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:46.841068 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:46.841068 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:49:46.841068 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:49:46.841068 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:49:46.841068 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:49:46.842067 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:46.842067 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:46.842067 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:46.842067 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:46.842067 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:46.842067 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:46.842067 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:46.842067 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:46.842067 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:46.842067 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:46.842067 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:46.842067 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:46.842067 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:46.842067 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:46.842067 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:46.842067 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:46.842067 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:49:46.842941 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:49:46.842941 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:46.842941 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:46.842941 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 276/558] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:49:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:46.843175 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63e8b32-c8d4-4ee2-bf33-5f763191ed1f None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:46.867371 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:46.900358 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118f9db2-ab74-4b91-91be-961122c49639 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:46.943837 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c947230-8ff5-4fe1-8705-eb1db94c4253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:46.944975 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c947230-8ff5-4fe1-8705-eb1db94c4253 None admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:46.945372 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c947230-8ff5-4fe1-8705-eb1db94c4253 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:46.945806 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c947230-8ff5-4fe1-8705-eb1db94c4253 None admin] PATH_INFO: `/v3/services` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:46.946882 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c947230-8ff5-4fe1-8705-eb1db94c4253 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:46.948805 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c947230-8ff5-4fe1-8705-eb1db94c4253 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:46.965969 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63e8b32-c8d4-4ee2-bf33-5f763191ed1f None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:47.005190 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ef11ce-e54e-4789-88e0-63147791aa96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:47.006235 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ef11ce-e54e-4789-88e0-63147791aa96 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:47.006664 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ef11ce-e54e-4789-88e0-63147791aa96 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:47.007156 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ef11ce-e54e-4789-88e0-63147791aa96 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:47.015897 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ef11ce-e54e-4789-88e0-63147791aa96 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:47.017558 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ef11ce-e54e-4789-88e0-63147791aa96 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:47.030710 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-96ef11ce-e54e-4789-88e0-63147791aa96 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 14 02:49:47.030710 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:47.030710 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:47.030710 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:47.030710 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:47.030710 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:47.030710 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:47.030710 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:47.030710 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:47.030710 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:47.030710 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:47.030710 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:47.030710 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 14 02:49:47.030710 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 14 02:49:47.030710 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 14 02:49:47.030710 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 14 02:49:47.031920 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:47.031920 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:47.031920 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 14 02:49:47.031920 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:49:47.031920 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 14 02:49:47.031920 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:49:47.031920 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:47.031920 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:47.031920 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:47.031920 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:47.031920 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:47.031920 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:47.031920 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:47.031920 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:47.031920 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:47.031920 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:47.031920 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:47.031920 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:47.033078 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:47.033078 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:47.033078 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 14 02:49:47.033078 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 14 02:49:47.033078 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 14 02:49:47.033078 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 14 02:49:47.033078 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 14 02:49:47.033078 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 14 02:49:47.033078 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 14 02:49:47.033078 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:47.035883 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 277/559] 10.222.0.39 () {56 vars in 1117 bytes} [Fri Oct 14 02:49:46 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:47.036872 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ef11ce-e54e-4789-88e0-63147791aa96 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:47.041838 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 283/560] 10.222.0.39 () {60 vars in 1141 bytes} [Fri Oct 14 02:49:46 2022] POST /identity/v3/services => generated 240 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:49:47.043031 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c947230-8ff5-4fe1-8705-eb1db94c4253 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:47.166648 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:47.193554 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:47.240177 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ef11ce-e54e-4789-88e0-63147791aa96 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:47.269293 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c947230-8ff5-4fe1-8705-eb1db94c4253 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:47.297307 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31df582f-8a7c-46cd-9167-6814e12b4630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:47.298024 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31df582f-8a7c-46cd-9167-6814e12b4630 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:47.298229 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31df582f-8a7c-46cd-9167-6814e12b4630 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:47.298507 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31df582f-8a7c-46cd-9167-6814e12b4630 None admin] PATH_INFO: `/v3/domains` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:47.299220 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31df582f-8a7c-46cd-9167-6814e12b4630 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:47.299526 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31df582f-8a7c-46cd-9167-6814e12b4630 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:47.301007 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31df582f-8a7c-46cd-9167-6814e12b4630 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:47.317801 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 278/561] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Oct 14 02:49:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:47.319879 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31df582f-8a7c-46cd-9167-6814e12b4630 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:47.327189 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6d2012-700b-4b88-b6b2-b380284134f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:47.327860 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6d2012-700b-4b88-b6b2-b380284134f6 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:47.328145 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6d2012-700b-4b88-b6b2-b380284134f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:47.328418 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6d2012-700b-4b88-b6b2-b380284134f6 None admin] PATH_INFO: `/v3/users` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:47.329156 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6d2012-700b-4b88-b6b2-b380284134f6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:47.329452 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6d2012-700b-4b88-b6b2-b380284134f6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:47.330919 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6d2012-700b-4b88-b6b2-b380284134f6 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:47.350009 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 284/562] 10.222.0.39 () {56 vars in 1149 bytes} [Fri Oct 14 02:49:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:47.359847 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6d2012-700b-4b88-b6b2-b380284134f6 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:47.457680 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31df582f-8a7c-46cd-9167-6814e12b4630 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:47.482481 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6d2012-700b-4b88-b6b2-b380284134f6 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:47.492719 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:47.512145 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23803f57-fc65-4598-a930-a67fc1fd8e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:47.513332 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23803f57-fc65-4598-a930-a67fc1fd8e34 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:47.514043 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23803f57-fc65-4598-a930-a67fc1fd8e34 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:47.514854 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23803f57-fc65-4598-a930-a67fc1fd8e34 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:47.519362 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2c21ab-1013-4cca-ab82-ba1b6f172c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:47.520655 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2c21ab-1013-4cca-ab82-ba1b6f172c89 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:47.521013 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2c21ab-1013-4cca-ab82-ba1b6f172c89 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:47.521265 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2c21ab-1013-4cca-ab82-ba1b6f172c89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:47.527117 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23803f57-fc65-4598-a930-a67fc1fd8e34 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:47.528806 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23803f57-fc65-4598-a930-a67fc1fd8e34 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:47.531929 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:47.534228 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2c21ab-1013-4cca-ab82-ba1b6f172c89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:47.535340 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2c21ab-1013-4cca-ab82-ba1b6f172c89 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:47.546467 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-23803f57-fc65-4598-a930-a67fc1fd8e34 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 14 02:49:47.546467 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:47.546467 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:47.546467 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:47.546467 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:47.546467 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:47.546467 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:47.546467 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:47.546467 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:47.546467 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:47.546467 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:47.546467 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:47.546467 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 14 02:49:47.546467 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 14 02:49:47.546467 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 14 02:49:47.546467 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 14 02:49:47.547436 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:47.547436 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:47.547436 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 14 02:49:47.547436 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:49:47.547436 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 14 02:49:47.547436 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:49:47.547436 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:47.547436 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:47.547436 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:47.547436 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:47.547436 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:47.547436 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:47.547436 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:47.547436 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:47.547436 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:47.547436 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:47.547436 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:47.547436 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:47.549670 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:47.549670 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:47.549670 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 14 02:49:47.549670 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 14 02:49:47.549670 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 14 02:49:47.549670 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 14 02:49:47.549670 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 14 02:49:47.549670 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 14 02:49:47.549670 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 14 02:49:47.549670 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:47.550778 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-0f2c21ab-1013-4cca-ab82-ba1b6f172c89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:47.550778 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:47.550778 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:47.550778 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:47.550778 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:47.550778 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:47.550778 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:47.550778 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:47.550778 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:47.550778 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:47.550778 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:47.550778 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:47.550778 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:49:47.550778 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:49:47.550778 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:49:47.550778 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:49:47.551694 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:47.551694 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:47.551694 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:47.551694 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:47.551694 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:47.551694 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:47.551694 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:47.551694 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:47.551694 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:47.551694 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:47.551694 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:47.551694 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:47.551694 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:47.551694 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:47.551694 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:47.551694 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:47.551694 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:49:47.552588 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:49:47.552588 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:47.552588 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:49:47.552588 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 285/563] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:49:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:47.562086 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 279/564] 10.222.0.39 () {56 vars in 1093 bytes} [Fri Oct 14 02:49:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:47.564803 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2c21ab-1013-4cca-ab82-ba1b6f172c89 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:47.567282 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23803f57-fc65-4598-a930-a67fc1fd8e34 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:47.679055 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2c21ab-1013-4cca-ab82-ba1b6f172c89 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:47.719018 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf81839-ebeb-4fff-8ece-9dac08c2aac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:47.720019 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf81839-ebeb-4fff-8ece-9dac08c2aac8 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:47.720513 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf81839-ebeb-4fff-8ece-9dac08c2aac8 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:47.720990 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf81839-ebeb-4fff-8ece-9dac08c2aac8 None admin] PATH_INFO: `/v3/domains` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:47.721894 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf81839-ebeb-4fff-8ece-9dac08c2aac8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:47.722411 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf81839-ebeb-4fff-8ece-9dac08c2aac8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:47.724995 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf81839-ebeb-4fff-8ece-9dac08c2aac8 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:47.740735 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 286/565] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Oct 14 02:49:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:47.747832 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf81839-ebeb-4fff-8ece-9dac08c2aac8 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:47.774413 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:47.823732 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23803f57-fc65-4598-a930-a67fc1fd8e34 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:47.864287 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf81839-ebeb-4fff-8ece-9dac08c2aac8 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:47.889975 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c02cbc4-50fe-4d75-8fa8-f4098b9275e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:47.890533 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c02cbc4-50fe-4d75-8fa8-f4098b9275e1 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:47.890823 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c02cbc4-50fe-4d75-8fa8-f4098b9275e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:47.891098 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c02cbc4-50fe-4d75-8fa8-f4098b9275e1 None admin] PATH_INFO: `/v3/users` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:47.891833 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c02cbc4-50fe-4d75-8fa8-f4098b9275e1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:47.892207 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c02cbc4-50fe-4d75-8fa8-f4098b9275e1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:47.893571 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c02cbc4-50fe-4d75-8fa8-f4098b9275e1 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:47.907430 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c449d461-69fd-42bf-8d3d-a6f5ec5a9ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:47.909166 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c449d461-69fd-42bf-8d3d-a6f5ec5a9ab2 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:47.909687 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c449d461-69fd-42bf-8d3d-a6f5ec5a9ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:47.910442 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c449d461-69fd-42bf-8d3d-a6f5ec5a9ab2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:47.912547 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 280/566] 10.222.0.39 () {56 vars in 1133 bytes} [Fri Oct 14 02:49:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:47.917378 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ef4e5c-c8bf-44f9-b0da-b188e9b461b4 None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:47.917378 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ef4e5c-c8bf-44f9-b0da-b188e9b461b4 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:47.917378 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ef4e5c-c8bf-44f9-b0da-b188e9b461b4 None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:47.920072 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 281/567] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:49:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:49:47.920873 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ef4e5c-c8bf-44f9-b0da-b188e9b461b4 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:47.924062 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c449d461-69fd-42bf-8d3d-a6f5ec5a9ab2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:47.927276 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c449d461-69fd-42bf-8d3d-a6f5ec5a9ab2 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:47.942264 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-c449d461-69fd-42bf-8d3d-a6f5ec5a9ab2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:49:47.942264 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:47.942264 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:47.942264 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:47.942264 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:47.942264 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:47.942264 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:47.942264 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:47.942264 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:47.942264 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:47.942264 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:47.942264 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:47.942264 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 14 02:49:47.942264 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 14 02:49:47.942264 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 14 02:49:47.943180 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 14 02:49:47.943180 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:47.943180 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:47.943180 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:47.943180 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:47.943180 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:47.943180 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:47.943180 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:47.943180 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:47.943180 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:47.943180 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:47.943180 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:47.943180 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:47.943180 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:47.943180 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:47.943180 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:47.943180 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:47.944076 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 14 02:49:47.944076 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 14 02:49:47.944076 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 14 02:49:47.944076 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 14 02:49:47.944076 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 14 02:49:47.944076 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 14 02:49:47.944076 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:49:47.944076 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:49:47.946516 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 287/568] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Oct 14 02:49:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:47.951291 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3281850-5f95-4727-a1d2-837d8ef4f5ca None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:47.952582 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3281850-5f95-4727-a1d2-837d8ef4f5ca None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:47.953948 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3281850-5f95-4727-a1d2-837d8ef4f5ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:47.972417 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-e3281850-5f95-4727-a1d2-837d8ef4f5ca None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:49:47.973561 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-e3281850-5f95-4727-a1d2-837d8ef4f5ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:49:47.998952 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:48.056681 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 288/569] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:49:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:49:48.057988 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3281850-5f95-4727-a1d2-837d8ef4f5ca None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:48.070459 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ef4e5c-c8bf-44f9-b0da-b188e9b461b4 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:48.072255 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:48.141878 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658ba788-3f5a-4203-93be-d3259e28996c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:48.142991 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658ba788-3f5a-4203-93be-d3259e28996c None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:48.142991 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658ba788-3f5a-4203-93be-d3259e28996c None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:48.143163 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658ba788-3f5a-4203-93be-d3259e28996c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:48.178129 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3281850-5f95-4727-a1d2-837d8ef4f5ca None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:48.218571 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b686c0-bfcf-4b73-9ff4-dfb3c296125b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:48.219540 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b686c0-bfcf-4b73-9ff4-dfb3c296125b None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:48.220039 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b686c0-bfcf-4b73-9ff4-dfb3c296125b None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:48.220596 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b686c0-bfcf-4b73-9ff4-dfb3c296125b None admin] PATH_INFO: `/v3/projects` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:48.221486 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b686c0-bfcf-4b73-9ff4-dfb3c296125b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:48.222050 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b686c0-bfcf-4b73-9ff4-dfb3c296125b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:48.225101 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b686c0-bfcf-4b73-9ff4-dfb3c296125b None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:48.239759 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 289/570] 10.222.0.39 () {56 vars in 1148 bytes} [Fri Oct 14 02:49:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:48.255672 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b686c0-bfcf-4b73-9ff4-dfb3c296125b None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:48.269811 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658ba788-3f5a-4203-93be-d3259e28996c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:48.269811 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658ba788-3f5a-4203-93be-d3259e28996c None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:48.325571 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:48.350155 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:48.380066 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 282/571] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Oct 14 02:49:47 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 14 02:49:48.385413 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658ba788-3f5a-4203-93be-d3259e28996c None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:48.419772 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b686c0-bfcf-4b73-9ff4-dfb3c296125b None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:48.481971 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dbe502-d91f-4198-8f05-9e14e58e95e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:48.482646 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dbe502-d91f-4198-8f05-9e14e58e95e0 None admin] REQUEST_METHOD: `PUT` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:48.482831 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dbe502-d91f-4198-8f05-9e14e58e95e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:48.483116 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dbe502-d91f-4198-8f05-9e14e58e95e0 None admin] PATH_INFO: `/v3/projects/dac9f8cd285c44e889a621b8b7fe479e/users/44b37050076b42e6af1a67bd7d6d06e1/roles/9c7226aa19e742759967212b7291782b` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:48.498027 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dbe502-d91f-4198-8f05-9e14e58e95e0 None admin] RBAC: Authorizing `identity:create_grant(project_id=dac9f8cd285c44e889a621b8b7fe479e, user_id=44b37050076b42e6af1a67bd7d6d06e1, role_id=9c7226aa19e742759967212b7291782b)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:48.499477 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dbe502-d91f-4198-8f05-9e14e58e95e0 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:48.532087 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658ba788-3f5a-4203-93be-d3259e28996c None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:48.564571 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3a61e1-2b74-4ec4-bedb-20e44557f2b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:48.565192 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3a61e1-2b74-4ec4-bedb-20e44557f2b5 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:48.565476 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3a61e1-2b74-4ec4-bedb-20e44557f2b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:48.565748 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3a61e1-2b74-4ec4-bedb-20e44557f2b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:48.573004 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 290/572] 10.222.0.39 () {58 vars in 1439 bytes} [Fri Oct 14 02:49:48 2022] PUT /identity/v3/projects/dac9f8cd285c44e889a621b8b7fe479e/users/44b37050076b42e6af1a67bd7d6d06e1/roles/9c7226aa19e742759967212b7291782b => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 02:49:48.574268 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3581a5-a55a-4896-9b9c-88a8052115e6 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:48.574553 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3581a5-a55a-4896-9b9c-88a8052115e6 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:48.574825 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3581a5-a55a-4896-9b9c-88a8052115e6 None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:48.576082 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 291/573] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:49:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:49:48.582685 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3581a5-a55a-4896-9b9c-88a8052115e6 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:48.584124 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3a61e1-2b74-4ec4-bedb-20e44557f2b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:48.586441 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3a61e1-2b74-4ec4-bedb-20e44557f2b5 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:48.610028 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-bb3a61e1-2b74-4ec4-bedb-20e44557f2b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:48.610028 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:48.610028 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:48.610028 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:48.610028 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:48.610028 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:48.610028 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:48.610028 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:48.610028 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:48.610028 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:48.610028 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:48.610028 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:48.610028 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:49:48.610028 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:49:48.610028 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:49:48.610028 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:49:48.611158 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:48.611158 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:48.611158 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:48.611158 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:48.611158 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:48.611158 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:48.611158 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:48.611158 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:48.611158 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:48.611158 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:48.611158 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:48.611158 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:48.611158 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:48.611158 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:48.611158 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:48.611158 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:48.611158 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:49:48.612051 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:49:48.612051 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:48.612051 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:48.614500 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 283/574] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:49:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:48.616927 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817d40cd-0c6d-46ec-af2b-d0942b379e17 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:48.617402 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817d40cd-0c6d-46ec-af2b-d0942b379e17 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:48.617849 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817d40cd-0c6d-46ec-af2b-d0942b379e17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:48.644166 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-817d40cd-0c6d-46ec-af2b-d0942b379e17 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:49:48.648158 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-817d40cd-0c6d-46ec-af2b-d0942b379e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:49:48.648158 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:48.728095 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 284/575] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:49:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:49:48.729484 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817d40cd-0c6d-46ec-af2b-d0942b379e17 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:48.749867 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3581a5-a55a-4896-9b9c-88a8052115e6 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:48.784831 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:48.804469 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea93d59-a203-427e-9a38-3e6f12e6696c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:48.805652 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea93d59-a203-427e-9a38-3e6f12e6696c None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:48.806085 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea93d59-a203-427e-9a38-3e6f12e6696c None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:48.806605 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea93d59-a203-427e-9a38-3e6f12e6696c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:48.807503 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea93d59-a203-427e-9a38-3e6f12e6696c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:48.809027 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea93d59-a203-427e-9a38-3e6f12e6696c None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:48.821979 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-cea93d59-a203-427e-9a38-3e6f12e6696c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 14 02:49:48.821979 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:48.821979 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 14 02:49:48.821979 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 14 02:49:48.821979 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 14 02:49:48.821979 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 14 02:49:48.821979 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 14 02:49:48.821979 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:49:48.821979 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 14 02:49:48.821979 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:49:48.821979 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 02:49:48.821979 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:49:48.821979 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:48.821979 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:48.821979 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:48.821979 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:48.821979 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:48.825193 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:48.825193 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:48.825193 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:48.825193 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:48.825193 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:48.825193 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:48.825193 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 14 02:49:48.825193 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 14 02:49:48.825193 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 14 02:49:48.825193 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 14 02:49:48.825193 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:48.825193 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:48.825193 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:48.825193 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:48.825193 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:48.825193 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:48.826274 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:48.826274 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:48.826274 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:48.826274 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:48.826274 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:48.826274 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:48.826274 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:48.826274 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:48.826274 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:48.826274 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:48.826274 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 14 02:49:48.826274 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:49:48.826274 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 14 02:49:48.826274 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:49:48.826274 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 292/576] 10.222.0.39 () {56 vars in 1129 bytes} [Fri Oct 14 02:49:48 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:48.829644 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea93d59-a203-427e-9a38-3e6f12e6696c None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:48.853364 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817d40cd-0c6d-46ec-af2b-d0942b379e17 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:48.902737 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0fef84-53d5-4136-9c4a-5d430254150c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:48.904353 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0fef84-53d5-4136-9c4a-5d430254150c None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:48.904787 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0fef84-53d5-4136-9c4a-5d430254150c None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:48.905211 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0fef84-53d5-4136-9c4a-5d430254150c None admin] PATH_INFO: `/v3/domains` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:48.906034 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0fef84-53d5-4136-9c4a-5d430254150c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:48.906661 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0fef84-53d5-4136-9c4a-5d430254150c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:48.909674 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0fef84-53d5-4136-9c4a-5d430254150c None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:48.923257 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:48.932625 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 285/577] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Oct 14 02:49:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:48.939144 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0fef84-53d5-4136-9c4a-5d430254150c None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:49.011679 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea93d59-a203-427e-9a38-3e6f12e6696c None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:49.061118 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d1cf3c-76a4-4fab-9d89-96d60794a467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:49.061739 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d1cf3c-76a4-4fab-9d89-96d60794a467 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:49.062417 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d1cf3c-76a4-4fab-9d89-96d60794a467 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:49.062700 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d1cf3c-76a4-4fab-9d89-96d60794a467 None admin] PATH_INFO: `/v3/services` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:49.063334 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d1cf3c-76a4-4fab-9d89-96d60794a467 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:49.063638 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d1cf3c-76a4-4fab-9d89-96d60794a467 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:49.065534 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d1cf3c-76a4-4fab-9d89-96d60794a467 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:49.069781 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0fef84-53d5-4136-9c4a-5d430254150c None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:49.077854 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 293/578] 10.222.0.39 () {56 vars in 1124 bytes} [Fri Oct 14 02:49:48 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:49.079010 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d1cf3c-76a4-4fab-9d89-96d60794a467 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:49.110376 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affd0a8d-dd13-4528-a79a-0011ba4849a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:49.111021 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affd0a8d-dd13-4528-a79a-0011ba4849a5 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:49.111310 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affd0a8d-dd13-4528-a79a-0011ba4849a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:49.111615 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affd0a8d-dd13-4528-a79a-0011ba4849a5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:49.119727 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affd0a8d-dd13-4528-a79a-0011ba4849a5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:49.121187 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affd0a8d-dd13-4528-a79a-0011ba4849a5 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:49.133855 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-affd0a8d-dd13-4528-a79a-0011ba4849a5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:49:49.133855 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:49.133855 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:49.133855 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:49.133855 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:49.133855 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:49.133855 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:49.133855 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:49.133855 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:49.133855 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:49.133855 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:49.133855 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:49.133855 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 14 02:49:49.133855 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 14 02:49:49.133855 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 14 02:49:49.134803 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 14 02:49:49.134803 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:49.134803 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:49.134803 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:49.134803 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:49.134803 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:49.134803 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:49.134803 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:49.134803 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:49.134803 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:49.134803 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:49.134803 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:49.134803 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:49.134803 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:49.134803 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:49.134803 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:49.134803 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:49.135949 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 14 02:49:49.135949 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 14 02:49:49.135949 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 14 02:49:49.135949 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 14 02:49:49.135949 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 14 02:49:49.135949 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 14 02:49:49.135949 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:49:49.135949 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:49.135949 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 286/579] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Oct 14 02:49:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:49.136510 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affd0a8d-dd13-4528-a79a-0011ba4849a5 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:49.141550 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:49.201059 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:49.227848 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d1cf3c-76a4-4fab-9d89-96d60794a467 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:49.255486 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affd0a8d-dd13-4528-a79a-0011ba4849a5 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:49.265950 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d0494c-6bef-471d-a02f-4bc989cd50a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:49.266790 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d0494c-6bef-471d-a02f-4bc989cd50a9 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:49.267280 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d0494c-6bef-471d-a02f-4bc989cd50a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:49.267628 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d0494c-6bef-471d-a02f-4bc989cd50a9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:49.278811 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d0494c-6bef-471d-a02f-4bc989cd50a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:49:49.284484 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-95d0494c-6bef-471d-a02f-4bc989cd50a9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:49:49.284484 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:49.284484 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:49.284484 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:49.284484 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:49.284484 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:49.284484 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:49.284484 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:49.284484 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:49.284484 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:49.284484 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:49.284484 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:49.284484 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:49:49.284484 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:49:49.284484 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:49:49.284484 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:49:49.285517 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:49:49.285517 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:49:49.285517 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:49:49.285517 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:49:49.285517 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:49.285517 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:49.285517 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:49.285517 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:49.285517 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:49.285517 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:49.285517 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:49.285517 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:49.285517 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:49.285517 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:49.285517 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:49.285517 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:49.286426 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:49.286426 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:49.286426 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:49.286426 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:49.286426 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:49:49.286426 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:49:49.286426 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:49:49.286426 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:49:49.286426 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:49:49.286426 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:49:49.286426 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:49:49.286426 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:49:49.286426 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 294/580] 10.222.0.39 () {56 vars in 1096 bytes} [Fri Oct 14 02:49:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:49.287115 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d0494c-6bef-471d-a02f-4bc989cd50a9 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:49.309028 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d654779c-9f42-486c-a709-c069aa3480d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:49.309960 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d654779c-9f42-486c-a709-c069aa3480d3 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:49.310428 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d654779c-9f42-486c-a709-c069aa3480d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:49.311106 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d654779c-9f42-486c-a709-c069aa3480d3 None admin] PATH_INFO: `/v3/services` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:49.312303 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d654779c-9f42-486c-a709-c069aa3480d3 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:49.312801 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d654779c-9f42-486c-a709-c069aa3480d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:49.314620 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d654779c-9f42-486c-a709-c069aa3480d3 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:49.323073 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 287/581] 10.222.0.39 () {56 vars in 1124 bytes} [Fri Oct 14 02:49:49 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:49.324219 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d654779c-9f42-486c-a709-c069aa3480d3 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:49.426145 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d0494c-6bef-471d-a02f-4bc989cd50a9 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:49.435988 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:49.442890 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d654779c-9f42-486c-a709-c069aa3480d3 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:49.463860 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c648e58a-5eda-4d7f-8216-65fb5be9941a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:49.464752 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c648e58a-5eda-4d7f-8216-65fb5be9941a None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:49.465216 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c648e58a-5eda-4d7f-8216-65fb5be9941a None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:49.465721 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c648e58a-5eda-4d7f-8216-65fb5be9941a None admin] PATH_INFO: `/v3/projects` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:49.466718 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:49.467118 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c648e58a-5eda-4d7f-8216-65fb5be9941a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:49.468181 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c648e58a-5eda-4d7f-8216-65fb5be9941a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:49.469816 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c648e58a-5eda-4d7f-8216-65fb5be9941a None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:49.479559 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df9e1b4-b58f-4ae9-8c18-56e63ec851b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:49.482797 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df9e1b4-b58f-4ae9-8c18-56e63ec851b9 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:49.482797 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df9e1b4-b58f-4ae9-8c18-56e63ec851b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:49.482797 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df9e1b4-b58f-4ae9-8c18-56e63ec851b9 None admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:49.482797 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df9e1b4-b58f-4ae9-8c18-56e63ec851b9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:49.482797 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df9e1b4-b58f-4ae9-8c18-56e63ec851b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:49.483152 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df9e1b4-b58f-4ae9-8c18-56e63ec851b9 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:49.486146 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 295/582] 10.222.0.39 () {56 vars in 1148 bytes} [Fri Oct 14 02:49:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:49.487298 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c648e58a-5eda-4d7f-8216-65fb5be9941a None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:49.498368 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 288/583] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Oct 14 02:49:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:49.498368 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df9e1b4-b58f-4ae9-8c18-56e63ec851b9 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:49.607902 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df9e1b4-b58f-4ae9-8c18-56e63ec851b9 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:49.642470 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c648e58a-5eda-4d7f-8216-65fb5be9941a None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:49.646207 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c28307-1e01-4f25-ae58-e3c87d9809a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:49.647195 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c28307-1e01-4f25-ae58-e3c87d9809a4 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:49.647742 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c28307-1e01-4f25-ae58-e3c87d9809a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:49.648230 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c28307-1e01-4f25-ae58-e3c87d9809a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:49.649189 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c28307-1e01-4f25-ae58-e3c87d9809a4 None admin] RBAC: Adding query filter params (role.id=9e86d182c0594e0785b72ba5f390a114, scope.project.id=dac9f8cd285c44e889a621b8b7fe479e, user.id=a6b07bf0faca47b28b50990d1720853f) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:49.649726 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c28307-1e01-4f25-ae58-e3c87d9809a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:49.651396 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c28307-1e01-4f25-ae58-e3c87d9809a4 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:49.658402 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 289/584] 10.222.0.39 () {56 vars in 1374 bytes} [Fri Oct 14 02:49:49 2022] GET /identity/v3/role_assignments?user.id=a6b07bf0faca47b28b50990d1720853f&scope.project.id=dac9f8cd285c44e889a621b8b7fe479e&role.id=9e86d182c0594e0785b72ba5f390a114 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:49.659611 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c28307-1e01-4f25-ae58-e3c87d9809a4 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:49.697404 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05422009-6634-4c93-a4ec-6804acdba118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:49.698099 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05422009-6634-4c93-a4ec-6804acdba118 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:49.698386 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05422009-6634-4c93-a4ec-6804acdba118 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:49.698666 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05422009-6634-4c93-a4ec-6804acdba118 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:49.699434 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05422009-6634-4c93-a4ec-6804acdba118 None admin] RBAC: Adding query filter params (interface=public, service_id=7ef02f0b36be4342bc14b9616868accc, region_id=RegionOne) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:49.701192 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05422009-6634-4c93-a4ec-6804acdba118 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:49.703262 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05422009-6634-4c93-a4ec-6804acdba118 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:49.712296 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 296/585] 10.222.0.39 () {56 vars in 1251 bytes} [Fri Oct 14 02:49:49 2022] GET /identity/v3/endpoints?service_id=7ef02f0b36be4342bc14b9616868accc&interface=public®ion_id=RegionOne => generated 194 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:49.713660 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457fe743-6743-4360-a19c-7f15f85dcdb3 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:49.713806 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457fe743-6743-4360-a19c-7f15f85dcdb3 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:49.718831 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457fe743-6743-4360-a19c-7f15f85dcdb3 None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:49.720346 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 297/586] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:49:49 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:49:49.721346 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457fe743-6743-4360-a19c-7f15f85dcdb3 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:49.737865 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:49.770101 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:49.799942 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c28307-1e01-4f25-ae58-e3c87d9809a4 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:49.842937 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db73f4a1-f89b-4b78-b04e-a34b6999fe27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:49.843870 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db73f4a1-f89b-4b78-b04e-a34b6999fe27 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:49.844303 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db73f4a1-f89b-4b78-b04e-a34b6999fe27 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:49.844725 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db73f4a1-f89b-4b78-b04e-a34b6999fe27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:49.858389 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db73f4a1-f89b-4b78-b04e-a34b6999fe27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:49.859866 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db73f4a1-f89b-4b78-b04e-a34b6999fe27 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:49.865796 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457fe743-6743-4360-a19c-7f15f85dcdb3 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:49.880124 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-db73f4a1-f89b-4b78-b04e-a34b6999fe27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:49.880124 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:49.880124 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:49.880124 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:49.880124 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:49.880124 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:49.880124 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:49.880124 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:49.880124 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:49.880124 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:49.880124 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:49.880124 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:49.880124 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:49:49.880124 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:49:49.880124 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:49:49.880124 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:49:49.881152 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:49.881152 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:49.881152 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:49.881152 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:49.881152 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:49.881152 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:49.881152 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:49.881152 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:49.881152 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:49.881152 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:49.881152 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:49.881152 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:49.881152 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:49.881152 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:49.881152 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:49.881152 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:49.881152 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:49:49.882071 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:49:49.882071 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:49.882071 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:49.884190 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 290/587] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:49:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:49.885556 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e75850-a5d1-4e6e-9281-cab3578fb14d None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:49.886049 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e75850-a5d1-4e6e-9281-cab3578fb14d None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:49.886425 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e75850-a5d1-4e6e-9281-cab3578fb14d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:49.909134 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0985e1fa-ad9e-4caa-8f69-019c74b1a52b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:49.909345 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-51e75850-a5d1-4e6e-9281-cab3578fb14d None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:49:49.909465 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0985e1fa-ad9e-4caa-8f69-019c74b1a52b None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:49.909465 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0985e1fa-ad9e-4caa-8f69-019c74b1a52b None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:49.909465 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0985e1fa-ad9e-4caa-8f69-019c74b1a52b None admin] PATH_INFO: `/v3/services` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:49.909697 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-51e75850-a5d1-4e6e-9281-cab3578fb14d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:49:49.910016 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0985e1fa-ad9e-4caa-8f69-019c74b1a52b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:49.911644 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0985e1fa-ad9e-4caa-8f69-019c74b1a52b None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:49.919359 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 298/588] 10.222.0.39 () {56 vars in 1087 bytes} [Fri Oct 14 02:49:49 2022] GET /identity/v3/services => generated 772 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:49.920453 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0985e1fa-ad9e-4caa-8f69-019c74b1a52b None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:50.040405 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 291/589] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:49:49 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:49:50.111110 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:50.119069 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0985e1fa-ad9e-4caa-8f69-019c74b1a52b None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:50.157531 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7438bb-da75-46a1-8ea7-b92b3d46d031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:50.158351 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7438bb-da75-46a1-8ea7-b92b3d46d031 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:50.158714 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7438bb-da75-46a1-8ea7-b92b3d46d031 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:50.159049 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7438bb-da75-46a1-8ea7-b92b3d46d031 None admin] PATH_INFO: `/v3/domains` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:50.159750 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7438bb-da75-46a1-8ea7-b92b3d46d031 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:50.160145 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7438bb-da75-46a1-8ea7-b92b3d46d031 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:50.161594 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7438bb-da75-46a1-8ea7-b92b3d46d031 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:50.173438 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 299/590] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Oct 14 02:49:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:50.179621 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e75850-a5d1-4e6e-9281-cab3578fb14d None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:50.211657 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:50.285796 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e75850-a5d1-4e6e-9281-cab3578fb14d None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:50.323255 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a260a9e8-f426-40fa-89cf-8dffa784aab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:50.323894 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a260a9e8-f426-40fa-89cf-8dffa784aab0 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:50.324174 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a260a9e8-f426-40fa-89cf-8dffa784aab0 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:50.324445 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a260a9e8-f426-40fa-89cf-8dffa784aab0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:50.334164 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a260a9e8-f426-40fa-89cf-8dffa784aab0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:50.335613 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a260a9e8-f426-40fa-89cf-8dffa784aab0 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:50.347511 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-a260a9e8-f426-40fa-89cf-8dffa784aab0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 14 02:49:50.347511 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:50.347511 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:50.347511 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:50.347511 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:50.347511 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:50.347511 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:50.347511 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:50.347511 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:50.347511 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:50.347511 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:50.347511 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:50.347511 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 14 02:49:50.347511 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 14 02:49:50.347511 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 14 02:49:50.347511 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 14 02:49:50.348500 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:50.348500 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:50.348500 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 14 02:49:50.348500 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:49:50.348500 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 14 02:49:50.348500 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:49:50.348500 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:50.348500 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:50.348500 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:50.348500 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:50.348500 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:50.348500 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:50.348500 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:50.348500 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:50.348500 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:50.348500 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:50.348500 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:50.348500 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:50.349452 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:50.349452 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:50.349452 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 14 02:49:50.349452 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 14 02:49:50.349452 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 14 02:49:50.349452 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 14 02:49:50.349452 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 14 02:49:50.349452 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 14 02:49:50.349452 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 14 02:49:50.349452 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:50.349452 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 292/591] 10.222.0.39 () {56 vars in 1096 bytes} [Fri Oct 14 02:49:50 2022] GET /identity/v3/users/swift => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:50.355094 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7438bb-da75-46a1-8ea7-b92b3d46d031 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:50.394855 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a260a9e8-f426-40fa-89cf-8dffa784aab0 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:50.494637 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:50.506375 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a260a9e8-f426-40fa-89cf-8dffa784aab0 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:50.544040 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb57dc7-9b0f-472b-9b34-56daba1b7064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:50.545053 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb57dc7-9b0f-472b-9b34-56daba1b7064 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:50.545538 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb57dc7-9b0f-472b-9b34-56daba1b7064 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:50.546051 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb57dc7-9b0f-472b-9b34-56daba1b7064 None admin] PATH_INFO: `/v3/services/network` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:50.546843 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb57dc7-9b0f-472b-9b34-56daba1b7064 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:50.549449 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb57dc7-9b0f-472b-9b34-56daba1b7064 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:50.553807 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7438bb-da75-46a1-8ea7-b92b3d46d031 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:50.559425 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-dcb57dc7-9b0f-472b-9b34-56daba1b7064 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 14 02:49:50.559425 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:50.559425 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 14 02:49:50.559425 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 14 02:49:50.559425 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 14 02:49:50.559425 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 14 02:49:50.559425 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 14 02:49:50.559425 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:49:50.559425 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 14 02:49:50.559425 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:50.559425 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 02:49:50.559425 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:50.559425 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:50.559425 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:50.559425 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:50.559425 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:50.559425 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:50.560456 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:50.560456 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:50.560456 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:50.560456 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:50.560456 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:50.560456 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:50.560456 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 14 02:49:50.560456 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 14 02:49:50.560456 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 14 02:49:50.560456 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 14 02:49:50.560456 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:50.560456 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:50.560456 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:50.560456 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:50.560456 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:50.560456 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:50.561290 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:50.561290 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:50.561290 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:50.561290 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:50.561290 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:50.561290 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:50.561290 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:50.561290 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:50.561290 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:50.561290 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:50.561290 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 14 02:49:50.561290 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:49:50.561290 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 14 02:49:50.561290 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:50.564096 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 293/592] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Oct 14 02:49:50 2022] GET /identity/v3/services/network => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 02:49:50.570022 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb57dc7-9b0f-472b-9b34-56daba1b7064 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:50.630935 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6e0703-ca7d-49f9-bec4-ca22b4cf9d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:50.631558 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6e0703-ca7d-49f9-bec4-ca22b4cf9d4a None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:50.631838 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6e0703-ca7d-49f9-bec4-ca22b4cf9d4a None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:50.632112 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6e0703-ca7d-49f9-bec4-ca22b4cf9d4a None admin] PATH_INFO: `/v3/users` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:50.633127 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6e0703-ca7d-49f9-bec4-ca22b4cf9d4a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:50.633127 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6e0703-ca7d-49f9-bec4-ca22b4cf9d4a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:50.634581 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6e0703-ca7d-49f9-bec4-ca22b4cf9d4a None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:50.665434 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 300/593] 10.222.0.39 () {56 vars in 1135 bytes} [Fri Oct 14 02:49:50 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:50.670860 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6e0703-ca7d-49f9-bec4-ca22b4cf9d4a None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:50.686775 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:50.711978 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb57dc7-9b0f-472b-9b34-56daba1b7064 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:50.779803 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd2ad7e-738f-4225-bf5d-4a4595ed11e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:50.780816 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd2ad7e-738f-4225-bf5d-4a4595ed11e7 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:50.781332 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd2ad7e-738f-4225-bf5d-4a4595ed11e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:50.781867 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd2ad7e-738f-4225-bf5d-4a4595ed11e7 None admin] PATH_INFO: `/v3/services` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:50.782714 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd2ad7e-738f-4225-bf5d-4a4595ed11e7 None admin] RBAC: Adding query filter params (name=network) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:50.783228 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd2ad7e-738f-4225-bf5d-4a4595ed11e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:50.784807 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd2ad7e-738f-4225-bf5d-4a4595ed11e7 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:50.790267 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 294/594] 10.222.0.39 () {56 vars in 1112 bytes} [Fri Oct 14 02:49:50 2022] GET /identity/v3/services?name=network => generated 124 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:49:50.797820 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6e0703-ca7d-49f9-bec4-ca22b4cf9d4a None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:50.799073 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd2ad7e-738f-4225-bf5d-4a4595ed11e7 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:50.834780 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62cab8f-1985-4fa7-a5e7-acfb0e1404c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:50.836238 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:50.837400 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62cab8f-1985-4fa7-a5e7-acfb0e1404c4 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:50.838093 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62cab8f-1985-4fa7-a5e7-acfb0e1404c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:50.838589 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62cab8f-1985-4fa7-a5e7-acfb0e1404c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:50.948632 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62cab8f-1985-4fa7-a5e7-acfb0e1404c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:50.950052 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62cab8f-1985-4fa7-a5e7-acfb0e1404c4 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:50.974172 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:50.979884 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd2ad7e-738f-4225-bf5d-4a4595ed11e7 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:51.020445 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b25fda4-812b-4065-887c-802b5dbac21a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:51.021240 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b25fda4-812b-4065-887c-802b5dbac21a None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:51.021665 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b25fda4-812b-4065-887c-802b5dbac21a None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:51.022133 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b25fda4-812b-4065-887c-802b5dbac21a None admin] PATH_INFO: `/v3/services` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:51.022986 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b25fda4-812b-4065-887c-802b5dbac21a None admin] RBAC: Adding query filter params (type=network) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:51.023432 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b25fda4-812b-4065-887c-802b5dbac21a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:51.024947 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b25fda4-812b-4065-887c-802b5dbac21a None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:51.031549 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 295/595] 10.222.0.39 () {56 vars in 1112 bytes} [Fri Oct 14 02:49:50 2022] GET /identity/v3/services?type=network => generated 350 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:51.033372 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6f9b9e-ae5b-49a5-b7d2-895d5b0a1fa5 None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:51.033850 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6f9b9e-ae5b-49a5-b7d2-895d5b0a1fa5 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:51.034330 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6f9b9e-ae5b-49a5-b7d2-895d5b0a1fa5 None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:51.040247 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 296/596] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:49:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:49:51.041658 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6f9b9e-ae5b-49a5-b7d2-895d5b0a1fa5 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:51.096771 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 301/597] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Oct 14 02:49:50 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 14 02:49:51.098803 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f7a919-a56e-4f5a-b85b-3b866aaa8d01 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:51.099088 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f7a919-a56e-4f5a-b85b-3b866aaa8d01 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:51.099360 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f7a919-a56e-4f5a-b85b-3b866aaa8d01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:51.123576 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-f4f7a919-a56e-4f5a-b85b-3b866aaa8d01 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:49:51.126397 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-f4f7a919-a56e-4f5a-b85b-3b866aaa8d01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:49:51.144529 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:51.167515 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6f9b9e-ae5b-49a5-b7d2-895d5b0a1fa5 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:51.210057 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd1cec2-7a39-4318-9bfb-8151533f8a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:51.210469 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 302/598] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:49:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:49:51.210911 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd1cec2-7a39-4318-9bfb-8151533f8a51 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:51.211298 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd1cec2-7a39-4318-9bfb-8151533f8a51 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:51.211410 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f7a919-a56e-4f5a-b85b-3b866aaa8d01 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:51.211752 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd1cec2-7a39-4318-9bfb-8151533f8a51 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:51.212615 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd1cec2-7a39-4318-9bfb-8151533f8a51 None admin] RBAC: Adding query filter params (interface=public, service_id=038f07e2863546898523ddf8688619b1, region_id=RegionOne) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:51.213045 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd1cec2-7a39-4318-9bfb-8151533f8a51 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:51.215762 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd1cec2-7a39-4318-9bfb-8151533f8a51 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:51.226265 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 297/599] 10.222.0.39 () {56 vars in 1251 bytes} [Fri Oct 14 02:49:51 2022] GET /identity/v3/endpoints?service_id=038f07e2863546898523ddf8688619b1&interface=public®ion_id=RegionOne => generated 194 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:51.235431 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd1cec2-7a39-4318-9bfb-8151533f8a51 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:51.284841 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:51.377906 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f7a919-a56e-4f5a-b85b-3b866aaa8d01 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:51.424336 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd1cec2-7a39-4318-9bfb-8151533f8a51 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:51.428580 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdaf86a-64ef-49ef-93c6-f4a65ea80763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:51.429213 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdaf86a-64ef-49ef-93c6-f4a65ea80763 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:51.429495 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdaf86a-64ef-49ef-93c6-f4a65ea80763 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:51.429787 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdaf86a-64ef-49ef-93c6-f4a65ea80763 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:51.446993 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdaf86a-64ef-49ef-93c6-f4a65ea80763 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:51.448474 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdaf86a-64ef-49ef-93c6-f4a65ea80763 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:51.473556 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-5bdaf86a-64ef-49ef-93c6-f4a65ea80763 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:51.473556 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:51.473556 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:51.473556 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:51.473556 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:51.473556 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:51.473556 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:51.473556 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:51.473556 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:51.473556 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:51.473556 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:51.473556 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:51.473556 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:49:51.473556 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:49:51.473556 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:49:51.473556 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:49:51.474668 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:51.474668 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:51.474668 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:51.474668 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:51.474668 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:51.474668 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:51.474668 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:51.474668 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:51.474668 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:51.474668 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:51.474668 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:51.474668 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:51.474668 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:51.474668 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:51.474668 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:51.474668 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:51.474668 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:49:51.475703 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:49:51.475703 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:51.475703 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:49:51.475703 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 303/600] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:49:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:51.479569 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdaf86a-64ef-49ef-93c6-f4a65ea80763 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:51.495385 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4478e66b-11e1-4cfe-b464-04e27f8a63ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:51.496001 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4478e66b-11e1-4cfe-b464-04e27f8a63ef None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:51.496287 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4478e66b-11e1-4cfe-b464-04e27f8a63ef None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:51.496568 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4478e66b-11e1-4cfe-b464-04e27f8a63ef None admin] PATH_INFO: `/v3/services` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:51.497187 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4478e66b-11e1-4cfe-b464-04e27f8a63ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:51.498591 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4478e66b-11e1-4cfe-b464-04e27f8a63ef None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:51.510766 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 298/601] 10.222.0.39 () {56 vars in 1087 bytes} [Fri Oct 14 02:49:51 2022] GET /identity/v3/services => generated 772 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:51.518695 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4478e66b-11e1-4cfe-b464-04e27f8a63ef None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:51.541389 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:51.608349 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:51.615196 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdaf86a-64ef-49ef-93c6-f4a65ea80763 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:51.658343 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb623335-bca8-42d1-9d08-82ae77771d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:51.659226 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb623335-bca8-42d1-9d08-82ae77771d57 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:51.659664 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb623335-bca8-42d1-9d08-82ae77771d57 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:51.660805 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb623335-bca8-42d1-9d08-82ae77771d57 None admin] PATH_INFO: `/v3/domains` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:51.661631 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb623335-bca8-42d1-9d08-82ae77771d57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:51.662136 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb623335-bca8-42d1-9d08-82ae77771d57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:51.663762 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb623335-bca8-42d1-9d08-82ae77771d57 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:51.678413 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 304/602] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Oct 14 02:49:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:49:51.683935 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb623335-bca8-42d1-9d08-82ae77771d57 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:51.792409 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4478e66b-11e1-4cfe-b464-04e27f8a63ef None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:51.804962 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb623335-bca8-42d1-9d08-82ae77771d57 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:51.829683 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05c84e7-409f-4d3e-9a94-73b69d29cc11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:51.830369 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05c84e7-409f-4d3e-9a94-73b69d29cc11 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:51.830664 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05c84e7-409f-4d3e-9a94-73b69d29cc11 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:51.830945 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05c84e7-409f-4d3e-9a94-73b69d29cc11 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:51.842804 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05c84e7-409f-4d3e-9a94-73b69d29cc11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:49:51.846334 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cc7d8c-2e40-4801-bdc5-1efaa87019c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:51.846970 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cc7d8c-2e40-4801-bdc5-1efaa87019c2 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:51.847247 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cc7d8c-2e40-4801-bdc5-1efaa87019c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:51.848079 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cc7d8c-2e40-4801-bdc5-1efaa87019c2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:51.849286 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-a05c84e7-409f-4d3e-9a94-73b69d29cc11 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 14 02:49:51.849286 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:51.849286 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:51.849286 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:51.849286 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:51.849286 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:51.849286 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:51.849286 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:51.849286 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:51.849286 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:51.849286 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:51.849286 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:51.849286 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:49:51.849286 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:49:51.849286 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:49:51.849286 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:49:51.850227 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:49:51.850227 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:49:51.850227 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:49:51.850227 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:49:51.850227 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:51.850227 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:51.850227 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:51.850227 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:51.850227 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:51.850227 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:51.850227 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:51.850227 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:51.850227 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:51.850227 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:51.850227 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:51.850227 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:51.851053 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:51.851053 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:51.851053 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:51.851053 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:51.851053 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:49:51.851053 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:49:51.851053 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:49:51.851053 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:49:51.851053 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:49:51.851053 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:49:51.851053 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 14 02:49:51.851053 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:51.851053 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 299/603] 10.222.0.39 () {56 vars in 1102 bytes} [Fri Oct 14 02:49:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:51.855753 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cc7d8c-2e40-4801-bdc5-1efaa87019c2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:51.857309 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cc7d8c-2e40-4801-bdc5-1efaa87019c2 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:51.858191 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05c84e7-409f-4d3e-9a94-73b69d29cc11 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:51.869210 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-a4cc7d8c-2e40-4801-bdc5-1efaa87019c2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:49:51.869210 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:51.869210 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:51.869210 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:51.869210 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:51.869210 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:51.869210 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:51.869210 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:51.869210 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:51.869210 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:51.869210 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:51.869210 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:51.869210 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 14 02:49:51.869210 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 14 02:49:51.869210 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 14 02:49:51.870173 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 14 02:49:51.870173 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:51.870173 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:51.870173 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:51.870173 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:51.870173 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:51.870173 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:51.870173 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:51.870173 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:51.870173 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:51.870173 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:51.870173 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:51.870173 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:51.870173 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:51.870173 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:51.870173 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:51.870173 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:51.871061 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 14 02:49:51.871061 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 14 02:49:51.871061 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 14 02:49:51.871061 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 14 02:49:51.871061 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 14 02:49:51.871061 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 14 02:49:51.871061 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:49:51.871061 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:49:51.871061 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 305/604] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Oct 14 02:49:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:51.879190 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cc7d8c-2e40-4801-bdc5-1efaa87019c2 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:51.919683 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:51.962696 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:52.019874 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05c84e7-409f-4d3e-9a94-73b69d29cc11 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:52.070906 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cc7d8c-2e40-4801-bdc5-1efaa87019c2 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:52.084074 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ccd58c-bdbb-409e-ad77-62fdfaa29ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:52.084696 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ccd58c-bdbb-409e-ad77-62fdfaa29ca6 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:52.084980 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ccd58c-bdbb-409e-ad77-62fdfaa29ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:52.085322 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ccd58c-bdbb-409e-ad77-62fdfaa29ca6 None admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:52.086027 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ccd58c-bdbb-409e-ad77-62fdfaa29ca6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:52.088112 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ccd58c-bdbb-409e-ad77-62fdfaa29ca6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:52.089504 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ccd58c-bdbb-409e-ad77-62fdfaa29ca6 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:52.102579 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 300/605] 10.222.0.39 () {56 vars in 1103 bytes} [Fri Oct 14 02:49:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:49:52.108655 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ccd58c-bdbb-409e-ad77-62fdfaa29ca6 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:52.136212 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50628314-0397-4402-ad15-12ce735271f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:52.136886 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50628314-0397-4402-ad15-12ce735271f3 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:52.137149 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50628314-0397-4402-ad15-12ce735271f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:52.137429 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50628314-0397-4402-ad15-12ce735271f3 None admin] PATH_INFO: `/v3/projects` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:52.139088 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50628314-0397-4402-ad15-12ce735271f3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:52.139398 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50628314-0397-4402-ad15-12ce735271f3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:52.140849 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50628314-0397-4402-ad15-12ce735271f3 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:52.167414 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 306/606] 10.222.0.39 () {56 vars in 1148 bytes} [Fri Oct 14 02:49:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:52.173008 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50628314-0397-4402-ad15-12ce735271f3 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:52.304531 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:52.307102 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ccd58c-bdbb-409e-ad77-62fdfaa29ca6 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:52.336341 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50628314-0397-4402-ad15-12ce735271f3 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:52.349363 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:52.356947 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf4a53a-643b-4f8b-aae3-69bb91926d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:52.361087 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf4a53a-643b-4f8b-aae3-69bb91926d05 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:52.361087 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf4a53a-643b-4f8b-aae3-69bb91926d05 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:52.361087 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf4a53a-643b-4f8b-aae3-69bb91926d05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:52.372031 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf4a53a-643b-4f8b-aae3-69bb91926d05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:52.374546 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf4a53a-643b-4f8b-aae3-69bb91926d05 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:52.383502 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f2c63f-fe16-420c-8a2d-4b81f048a259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:52.384114 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f2c63f-fe16-420c-8a2d-4b81f048a259 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:52.384396 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f2c63f-fe16-420c-8a2d-4b81f048a259 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:52.384669 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f2c63f-fe16-420c-8a2d-4b81f048a259 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:52.385390 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f2c63f-fe16-420c-8a2d-4b81f048a259 None admin] RBAC: Adding query filter params (role.id=9e86d182c0594e0785b72ba5f390a114, scope.project.id=dac9f8cd285c44e889a621b8b7fe479e, user.id=5c5e07da64804243bc18862072b027a1) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:52.385697 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f2c63f-fe16-420c-8a2d-4b81f048a259 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:52.388159 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f2c63f-fe16-420c-8a2d-4b81f048a259 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:52.394528 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-9cf4a53a-643b-4f8b-aae3-69bb91926d05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:52.394528 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:52.394528 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:52.394528 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:52.394528 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:52.394528 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:52.394528 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:52.394528 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:52.394528 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:52.394528 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:52.394528 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:52.394528 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:52.394528 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:49:52.394528 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:49:52.394528 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:49:52.394528 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:49:52.395533 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:52.395533 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:52.395533 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:52.395533 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:52.395533 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:52.395533 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:52.395533 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:52.395533 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:52.395533 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:52.395533 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:52.395533 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:52.395533 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:52.395533 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:52.395533 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:52.395533 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:52.395533 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:52.395533 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:49:52.396424 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:49:52.396424 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:52.396424 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:52.396596 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 307/607] 10.222.0.39 () {56 vars in 1374 bytes} [Fri Oct 14 02:49:52 2022] GET /identity/v3/role_assignments?user.id=5c5e07da64804243bc18862072b027a1&scope.project.id=dac9f8cd285c44e889a621b8b7fe479e&role.id=9e86d182c0594e0785b72ba5f390a114 => generated 608 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:52.396596 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42336bda-9425-4e5d-891c-ca0e197d2d60 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:52.396596 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42336bda-9425-4e5d-891c-ca0e197d2d60 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:52.396596 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42336bda-9425-4e5d-891c-ca0e197d2d60 None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:52.399935 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 301/608] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:49:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:52.401906 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 308/609] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:49:52 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:49:52.405349 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf4a53a-643b-4f8b-aae3-69bb91926d05 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:52.407949 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d647cd-a10f-4223-b02d-f0d4639e3ed4 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:52.408566 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d647cd-a10f-4223-b02d-f0d4639e3ed4 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:52.409082 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d647cd-a10f-4223-b02d-f0d4639e3ed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:52.428960 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-c7d647cd-a10f-4223-b02d-f0d4639e3ed4 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:49:52.430561 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-c7d647cd-a10f-4223-b02d-f0d4639e3ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:49:52.507421 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 309/610] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:49:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:49:52.554123 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf4a53a-643b-4f8b-aae3-69bb91926d05 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:52.603135 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1c0e36-61c2-4fe5-a4b1-1d210de71b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:52.604480 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1c0e36-61c2-4fe5-a4b1-1d210de71b1d None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:52.604999 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1c0e36-61c2-4fe5-a4b1-1d210de71b1d None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:52.605488 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1c0e36-61c2-4fe5-a4b1-1d210de71b1d None admin] PATH_INFO: `/v3/domains` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:52.606396 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1c0e36-61c2-4fe5-a4b1-1d210de71b1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:52.606946 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1c0e36-61c2-4fe5-a4b1-1d210de71b1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:52.608292 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c9b492-e8e1-4ac2-8f26-bbcdfc813990 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:52.608576 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c9b492-e8e1-4ac2-8f26-bbcdfc813990 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:52.608849 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c9b492-e8e1-4ac2-8f26-bbcdfc813990 None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:52.609243 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1c0e36-61c2-4fe5-a4b1-1d210de71b1d None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:52.616553 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 310/611] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:49:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 14 02:49:52.616728 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13aff49-65f4-4e85-9b87-9824fdbb690e None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:52.617033 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13aff49-65f4-4e85-9b87-9824fdbb690e None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:52.617203 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13aff49-65f4-4e85-9b87-9824fdbb690e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:52.633936 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 302/612] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Oct 14 02:49:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:49:52.639147 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1c0e36-61c2-4fe5-a4b1-1d210de71b1d None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:52.641473 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-e13aff49-65f4-4e85-9b87-9824fdbb690e None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:49:52.643935 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-e13aff49-65f4-4e85-9b87-9824fdbb690e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:49:52.691042 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:52.725386 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:52.734770 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 311/613] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:49:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 14 02:49:52.760505 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1c0e36-61c2-4fe5-a4b1-1d210de71b1d None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:52.806469 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00243b58-49d8-431a-b819-88ca14783ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:52.807424 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00243b58-49d8-431a-b819-88ca14783ba6 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:52.807848 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00243b58-49d8-431a-b819-88ca14783ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:52.808279 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00243b58-49d8-431a-b819-88ca14783ba6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:52.817974 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00243b58-49d8-431a-b819-88ca14783ba6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:52.819599 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00243b58-49d8-431a-b819-88ca14783ba6 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:52.833998 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-00243b58-49d8-431a-b819-88ca14783ba6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 14 02:49:52.833998 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:52.833998 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:52.833998 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:52.833998 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:52.833998 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:52.833998 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:52.833998 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:52.833998 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:52.833998 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:52.833998 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:52.833998 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:52.833998 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 14 02:49:52.833998 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 14 02:49:52.833998 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 14 02:49:52.833998 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 14 02:49:52.835034 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:52.835034 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:52.835034 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 14 02:49:52.835034 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:49:52.835034 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 14 02:49:52.835034 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:49:52.835034 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:52.835034 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:52.835034 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:52.835034 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:52.835034 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:52.835034 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:52.835034 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:52.835034 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:52.835034 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:52.835034 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:52.835034 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:52.835034 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:52.836074 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:52.836074 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:52.836074 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 14 02:49:52.836074 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 14 02:49:52.836074 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 14 02:49:52.836074 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 14 02:49:52.836074 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 14 02:49:52.836074 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 14 02:49:52.836074 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 14 02:49:52.836074 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:52.837981 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 303/614] 10.222.0.39 () {56 vars in 1117 bytes} [Fri Oct 14 02:49:52 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:52.843598 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13aff49-65f4-4e85-9b87-9824fdbb690e None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:52.847211 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00243b58-49d8-431a-b819-88ca14783ba6 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:52.970286 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00243b58-49d8-431a-b819-88ca14783ba6 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:52.982072 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:53.031973 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13aff49-65f4-4e85-9b87-9824fdbb690e None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:53.063015 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f46f60-4eac-4f6f-829f-8b0f8d076fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:53.063656 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f46f60-4eac-4f6f-829f-8b0f8d076fb4 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:53.063937 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f46f60-4eac-4f6f-829f-8b0f8d076fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:53.064211 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f46f60-4eac-4f6f-829f-8b0f8d076fb4 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:53.064793 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f46f60-4eac-4f6f-829f-8b0f8d076fb4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:53.066749 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f46f60-4eac-4f6f-829f-8b0f8d076fb4 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:53.072889 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dbb05d-9670-43f5-bd58-2280ba02975f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:53.073493 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dbb05d-9670-43f5-bd58-2280ba02975f None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:53.073795 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dbb05d-9670-43f5-bd58-2280ba02975f None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:53.074065 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dbb05d-9670-43f5-bd58-2280ba02975f None admin] PATH_INFO: `/v3/users` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:53.074787 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dbb05d-9670-43f5-bd58-2280ba02975f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:53.075095 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dbb05d-9670-43f5-bd58-2280ba02975f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:53.076516 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dbb05d-9670-43f5-bd58-2280ba02975f None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:53.090292 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-92f46f60-4eac-4f6f-829f-8b0f8d076fb4 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 14 02:49:53.090292 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:53.090292 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 14 02:49:53.090292 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 14 02:49:53.090292 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 14 02:49:53.090292 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 14 02:49:53.090292 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 14 02:49:53.090292 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:49:53.090292 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 14 02:49:53.090292 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:53.090292 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 02:49:53.090292 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:53.090292 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:53.090292 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:53.090292 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:53.090292 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:53.091304 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:53.091304 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:53.091304 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:53.091304 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:53.091304 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:53.091304 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:53.091304 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:53.091304 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 14 02:49:53.091304 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 14 02:49:53.091304 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 14 02:49:53.091304 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 14 02:49:53.091304 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:53.091304 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:53.091304 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:53.091304 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:53.091304 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:53.091304 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:53.092214 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:53.092214 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:53.092214 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:53.092214 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:53.092214 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:53.092214 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:53.092214 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:53.092214 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:53.092214 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:53.092214 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:53.092214 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 14 02:49:53.092214 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:49:53.092214 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 14 02:49:53.092214 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:53.092214 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 304/615] 10.222.0.39 () {56 vars in 1132 bytes} [Fri Oct 14 02:49:52 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:53.098873 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f46f60-4eac-4f6f-829f-8b0f8d076fb4 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:53.107636 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 312/616] 10.222.0.39 () {56 vars in 1149 bytes} [Fri Oct 14 02:49:52 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:53.116650 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dbb05d-9670-43f5-bd58-2280ba02975f None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:53.179709 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:53.223782 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dbb05d-9670-43f5-bd58-2280ba02975f None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:53.225604 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f46f60-4eac-4f6f-829f-8b0f8d076fb4 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:53.261240 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831f10c5-4802-4ff2-8d7c-eef5fd95511c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:53.261988 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831f10c5-4802-4ff2-8d7c-eef5fd95511c None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:53.262332 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831f10c5-4802-4ff2-8d7c-eef5fd95511c None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:53.262674 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831f10c5-4802-4ff2-8d7c-eef5fd95511c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:53.269476 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0987b670-98ab-4f35-be0d-51e192fabe87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:53.270131 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0987b670-98ab-4f35-be0d-51e192fabe87 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:53.270412 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0987b670-98ab-4f35-be0d-51e192fabe87 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:53.270736 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0987b670-98ab-4f35-be0d-51e192fabe87 None admin] PATH_INFO: `/v3/services` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:53.271898 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0987b670-98ab-4f35-be0d-51e192fabe87 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:53.272214 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0987b670-98ab-4f35-be0d-51e192fabe87 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:53.273603 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0987b670-98ab-4f35-be0d-51e192fabe87 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:53.279942 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 305/617] 10.222.0.39 () {56 vars in 1126 bytes} [Fri Oct 14 02:49:53 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:49:53.280846 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0987b670-98ab-4f35-be0d-51e192fabe87 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:53.316896 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:53.352333 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831f10c5-4802-4ff2-8d7c-eef5fd95511c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:53.355226 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831f10c5-4802-4ff2-8d7c-eef5fd95511c None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:53.394014 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0987b670-98ab-4f35-be0d-51e192fabe87 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:53.431555 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:53.433046 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558d4b6b-f33d-420c-90e1-d7e6df5dca7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:53.434356 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558d4b6b-f33d-420c-90e1-d7e6df5dca7b None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:53.434689 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558d4b6b-f33d-420c-90e1-d7e6df5dca7b None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:53.434966 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558d4b6b-f33d-420c-90e1-d7e6df5dca7b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:53.435550 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558d4b6b-f33d-420c-90e1-d7e6df5dca7b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:53.436925 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558d4b6b-f33d-420c-90e1-d7e6df5dca7b None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:53.447101 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-558d4b6b-f33d-420c-90e1-d7e6df5dca7b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 14 02:49:53.447101 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:53.447101 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 14 02:49:53.447101 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 14 02:49:53.447101 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 14 02:49:53.447101 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 14 02:49:53.447101 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 14 02:49:53.447101 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:49:53.447101 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 14 02:49:53.447101 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:53.447101 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 02:49:53.447101 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:53.447101 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:53.447101 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:53.447101 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:53.447101 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:53.447101 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:53.448268 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:53.448268 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:53.448268 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:53.448268 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:53.448268 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:53.448268 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:53.448268 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 14 02:49:53.448268 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 14 02:49:53.448268 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 14 02:49:53.448268 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 14 02:49:53.448268 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:53.448268 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:53.448268 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:53.448268 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:53.448268 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:53.448268 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:53.449325 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:53.449325 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:53.449325 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:53.449325 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:53.449325 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:53.449325 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:53.449325 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:53.449325 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:53.449325 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:53.449325 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:53.449325 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 14 02:49:53.449325 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:49:53.449325 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 14 02:49:53.449325 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:53.454146 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 306/618] 10.222.0.39 () {56 vars in 1129 bytes} [Fri Oct 14 02:49:53 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:53.454999 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558d4b6b-f33d-420c-90e1-d7e6df5dca7b None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:53.485893 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 313/619] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Oct 14 02:49:53 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 14 02:49:53.487107 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831f10c5-4802-4ff2-8d7c-eef5fd95511c None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:53.574035 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:53.580827 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558d4b6b-f33d-420c-90e1-d7e6df5dca7b None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:53.596028 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831f10c5-4802-4ff2-8d7c-eef5fd95511c None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:53.623573 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77c7b7d-96c4-445c-b990-474a61c62faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:53.624175 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77c7b7d-96c4-445c-b990-474a61c62faf None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:53.624450 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77c7b7d-96c4-445c-b990-474a61c62faf None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:53.624726 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77c7b7d-96c4-445c-b990-474a61c62faf None admin] PATH_INFO: `/v3/services` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:53.625510 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77c7b7d-96c4-445c-b990-474a61c62faf None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:53.625839 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77c7b7d-96c4-445c-b990-474a61c62faf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:53.627844 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77c7b7d-96c4-445c-b990-474a61c62faf None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:53.634713 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 307/620] 10.222.0.39 () {56 vars in 1126 bytes} [Fri Oct 14 02:49:53 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:53.635762 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77c7b7d-96c4-445c-b990-474a61c62faf None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:53.636554 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fc22c7-6801-4ad1-aae1-51766950991a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:53.637316 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fc22c7-6801-4ad1-aae1-51766950991a None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:53.637595 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fc22c7-6801-4ad1-aae1-51766950991a None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:53.637890 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fc22c7-6801-4ad1-aae1-51766950991a None admin] PATH_INFO: `/v3/services` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:53.638530 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fc22c7-6801-4ad1-aae1-51766950991a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:53.638834 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fc22c7-6801-4ad1-aae1-51766950991a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:53.640498 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fc22c7-6801-4ad1-aae1-51766950991a None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:53.653844 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 314/621] 10.222.0.39 () {56 vars in 1124 bytes} [Fri Oct 14 02:49:53 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:53.657238 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fc22c7-6801-4ad1-aae1-51766950991a None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:53.689555 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:53.824155 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77c7b7d-96c4-445c-b990-474a61c62faf None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:53.855632 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fc22c7-6801-4ad1-aae1-51766950991a None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:53.869513 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c278f928-dc31-4b7b-a127-961b09b88e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:53.870470 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c278f928-dc31-4b7b-a127-961b09b88e7a None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:53.870954 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c278f928-dc31-4b7b-a127-961b09b88e7a None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:53.871440 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c278f928-dc31-4b7b-a127-961b09b88e7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:53.883008 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c278f928-dc31-4b7b-a127-961b09b88e7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:53.884744 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c278f928-dc31-4b7b-a127-961b09b88e7a None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:53.892305 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ee65e1-9a17-4a16-b09f-5c69466bacfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:53.892969 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ee65e1-9a17-4a16-b09f-5c69466bacfd None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:53.893254 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ee65e1-9a17-4a16-b09f-5c69466bacfd None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:53.893526 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ee65e1-9a17-4a16-b09f-5c69466bacfd None admin] PATH_INFO: `/v3/services` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:53.895644 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ee65e1-9a17-4a16-b09f-5c69466bacfd None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:53.895955 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ee65e1-9a17-4a16-b09f-5c69466bacfd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:53.897332 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ee65e1-9a17-4a16-b09f-5c69466bacfd None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:53.901721 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-c278f928-dc31-4b7b-a127-961b09b88e7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:53.901721 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:53.901721 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:53.901721 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:53.901721 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:53.901721 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:53.901721 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:53.901721 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:53.901721 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:53.901721 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:53.901721 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:53.901721 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:53.901721 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:49:53.901721 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:49:53.901721 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:49:53.901721 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:49:53.902684 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:53.902684 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:53.902684 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:53.902684 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:53.902684 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:53.902684 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:53.902684 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:53.902684 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:53.902684 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:53.902684 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:53.902684 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:53.902684 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:53.902684 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:53.902684 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:53.902684 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:53.902684 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:53.902684 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:49:53.903605 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:49:53.903605 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:53.903605 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:53.903794 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 315/622] 10.222.0.39 () {56 vars in 1124 bytes} [Fri Oct 14 02:49:53 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:53.905433 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 308/623] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:49:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:53.908189 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ee65e1-9a17-4a16-b09f-5c69466bacfd None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:53.912428 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c278f928-dc31-4b7b-a127-961b09b88e7a None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:53.928627 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:54.025671 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:54.027279 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c278f928-dc31-4b7b-a127-961b09b88e7a None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:54.049169 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ee65e1-9a17-4a16-b09f-5c69466bacfd None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:54.066444 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0362afd-d2e9-4855-ba69-8b0ff8d5394d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:54.067302 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0362afd-d2e9-4855-ba69-8b0ff8d5394d None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:54.067609 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0362afd-d2e9-4855-ba69-8b0ff8d5394d None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:54.067888 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0362afd-d2e9-4855-ba69-8b0ff8d5394d None admin] PATH_INFO: `/v3/domains` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:54.068532 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0362afd-d2e9-4855-ba69-8b0ff8d5394d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:54.068840 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0362afd-d2e9-4855-ba69-8b0ff8d5394d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:54.070890 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0362afd-d2e9-4855-ba69-8b0ff8d5394d None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:54.087867 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 309/624] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Oct 14 02:49:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:54.088663 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deff7dc0-a8e8-45e5-967e-5891915ef7a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:54.089459 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deff7dc0-a8e8-45e5-967e-5891915ef7a2 None admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:54.089945 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deff7dc0-a8e8-45e5-967e-5891915ef7a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:54.090392 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deff7dc0-a8e8-45e5-967e-5891915ef7a2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:54.091428 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deff7dc0-a8e8-45e5-967e-5891915ef7a2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:54.093014 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deff7dc0-a8e8-45e5-967e-5891915ef7a2 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:54.093134 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0362afd-d2e9-4855-ba69-8b0ff8d5394d None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:54.186210 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:54.205303 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0362afd-d2e9-4855-ba69-8b0ff8d5394d None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:54.243951 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 316/625] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 02:49:53 2022] POST /identity/v3/endpoints => generated 346 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 02:49:54.270430 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fee378-bb06-440f-b811-d70335cc914e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:54.271152 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fee378-bb06-440f-b811-d70335cc914e None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:54.271440 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fee378-bb06-440f-b811-d70335cc914e None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:54.271716 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fee378-bb06-440f-b811-d70335cc914e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:54.281123 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fee378-bb06-440f-b811-d70335cc914e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:54.282808 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fee378-bb06-440f-b811-d70335cc914e None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:54.302638 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-01fee378-bb06-440f-b811-d70335cc914e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:49:54.302638 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:54.302638 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:54.302638 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:54.302638 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:54.302638 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:54.302638 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:54.302638 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:54.302638 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:54.302638 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:54.302638 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:54.302638 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:54.302638 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 14 02:49:54.302638 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 14 02:49:54.302638 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 14 02:49:54.303577 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 14 02:49:54.303577 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:54.303577 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:54.303577 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:54.303577 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:54.303577 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:54.303577 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:54.303577 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:54.303577 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:54.303577 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:54.303577 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:54.303577 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:54.303577 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:54.303577 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:54.303577 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:54.303577 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:54.303577 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:54.304465 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 14 02:49:54.304465 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 14 02:49:54.304465 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 14 02:49:54.304465 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 14 02:49:54.304465 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 14 02:49:54.304465 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 14 02:49:54.304465 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:49:54.304465 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:54.308259 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 310/626] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Oct 14 02:49:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 02:49:54.309649 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deff7dc0-a8e8-45e5-967e-5891915ef7a2 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:54.346817 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9093a6-cd5f-4e87-bc95-fc811804abc1 None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:54.347003 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9093a6-cd5f-4e87-bc95-fc811804abc1 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:54.347309 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9093a6-cd5f-4e87-bc95-fc811804abc1 None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:54.348768 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 311/627] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:49:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 14 02:49:54.360345 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6c13b7-1caa-4c56-80b9-d9d195a55dc7 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:54.360673 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6c13b7-1caa-4c56-80b9-d9d195a55dc7 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:54.360894 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6c13b7-1caa-4c56-80b9-d9d195a55dc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:54.392836 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-5d6c13b7-1caa-4c56-80b9-d9d195a55dc7 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:49:54.395214 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-5d6c13b7-1caa-4c56-80b9-d9d195a55dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:49:54.506856 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deff7dc0-a8e8-45e5-967e-5891915ef7a2 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:54.517966 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 312/628] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:49:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 14 02:49:54.552503 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580398c3-5ad1-42d9-a9a3-500cbd1ef13c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:54.553112 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580398c3-5ad1-42d9-a9a3-500cbd1ef13c None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:54.553391 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580398c3-5ad1-42d9-a9a3-500cbd1ef13c None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:54.553660 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580398c3-5ad1-42d9-a9a3-500cbd1ef13c None admin] PATH_INFO: `/v3/projects` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:54.554400 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580398c3-5ad1-42d9-a9a3-500cbd1ef13c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:54.554703 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580398c3-5ad1-42d9-a9a3-500cbd1ef13c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:54.556175 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580398c3-5ad1-42d9-a9a3-500cbd1ef13c None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:54.568944 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 317/629] 10.222.0.39 () {56 vars in 1148 bytes} [Fri Oct 14 02:49:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:54.574561 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6c13b7-1caa-4c56-80b9-d9d195a55dc7 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:54.640574 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:54.688842 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6c13b7-1caa-4c56-80b9-d9d195a55dc7 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:54.731380 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba180612-f06e-48df-b6ac-ba551bff54a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:54.732014 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba180612-f06e-48df-b6ac-ba551bff54a2 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:54.732325 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba180612-f06e-48df-b6ac-ba551bff54a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:54.732635 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba180612-f06e-48df-b6ac-ba551bff54a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:54.733417 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba180612-f06e-48df-b6ac-ba551bff54a2 None admin] RBAC: Adding query filter params (role.id=9c7226aa19e742759967212b7291782b, scope.project.id=dac9f8cd285c44e889a621b8b7fe479e, user.id=44b37050076b42e6af1a67bd7d6d06e1) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:54.733781 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba180612-f06e-48df-b6ac-ba551bff54a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:54.735341 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba180612-f06e-48df-b6ac-ba551bff54a2 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:54.746626 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 313/630] 10.222.0.39 () {56 vars in 1374 bytes} [Fri Oct 14 02:49:54 2022] GET /identity/v3/role_assignments?user.id=44b37050076b42e6af1a67bd7d6d06e1&scope.project.id=dac9f8cd285c44e889a621b8b7fe479e&role.id=9c7226aa19e742759967212b7291782b => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:54.844293 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580398c3-5ad1-42d9-a9a3-500cbd1ef13c None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:54.848379 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:54.920277 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a6f7f2-0b52-497c-93e8-5f159e58a936 None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:54.920550 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a6f7f2-0b52-497c-93e8-5f159e58a936 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:54.920772 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a6f7f2-0b52-497c-93e8-5f159e58a936 None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:54.922336 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 314/631] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:49:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:49:54.927662 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14fce06-12ac-455f-9f27-cdd642570dd5 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:54.927946 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14fce06-12ac-455f-9f27-cdd642570dd5 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:54.928224 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14fce06-12ac-455f-9f27-cdd642570dd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:54.961673 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-b14fce06-12ac-455f-9f27-cdd642570dd5 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:49:54.962920 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-b14fce06-12ac-455f-9f27-cdd642570dd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:49:54.993543 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580398c3-5ad1-42d9-a9a3-500cbd1ef13c None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:55.044898 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fca4a54-2bfa-4810-b926-10ea6fe8c604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:55.045811 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fca4a54-2bfa-4810-b926-10ea6fe8c604 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:55.046108 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fca4a54-2bfa-4810-b926-10ea6fe8c604 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:55.046795 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fca4a54-2bfa-4810-b926-10ea6fe8c604 None admin] PATH_INFO: `/v3/services/network` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:55.047405 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fca4a54-2bfa-4810-b926-10ea6fe8c604 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:55.049702 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fca4a54-2bfa-4810-b926-10ea6fe8c604 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:55.060247 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 315/632] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:49:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 14 02:49:55.062514 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-0fca4a54-2bfa-4810-b926-10ea6fe8c604 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 14 02:49:55.062514 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:55.062514 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 14 02:49:55.062514 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 14 02:49:55.062514 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 14 02:49:55.062514 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 14 02:49:55.062514 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 14 02:49:55.062514 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:49:55.062514 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 14 02:49:55.062514 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:49:55.062514 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 02:49:55.062514 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:49:55.062514 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:55.062514 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:55.062514 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:55.062514 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:55.062514 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:55.063794 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:55.063794 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:55.063794 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:55.063794 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:55.063794 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:55.063794 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:55.063794 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 14 02:49:55.063794 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 14 02:49:55.063794 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 14 02:49:55.063794 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 14 02:49:55.063794 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:55.063794 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:55.063794 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:55.063794 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:55.063794 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:55.063794 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:55.064739 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:55.064739 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:55.064739 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:55.064739 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:55.064739 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:55.064739 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:55.064739 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:55.064739 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:55.064739 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:55.064739 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:55.064739 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 14 02:49:55.064739 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:49:55.064739 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 14 02:49:55.064739 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:49:55.065538 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 318/633] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Oct 14 02:49:54 2022] GET /identity/v3/services/network => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:55.071163 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14fce06-12ac-455f-9f27-cdd642570dd5 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:55.130517 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:55.182276 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14fce06-12ac-455f-9f27-cdd642570dd5 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:55.227764 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbd857a-435e-481e-8edf-4d9d4c594320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:55.228754 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbd857a-435e-481e-8edf-4d9d4c594320 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:55.229486 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbd857a-435e-481e-8edf-4d9d4c594320 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:55.230009 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbd857a-435e-481e-8edf-4d9d4c594320 None admin] PATH_INFO: `/v3/services` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:55.230918 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbd857a-435e-481e-8edf-4d9d4c594320 None admin] RBAC: Adding query filter params (name=network) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:55.231449 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbd857a-435e-481e-8edf-4d9d4c594320 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:55.233024 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbd857a-435e-481e-8edf-4d9d4c594320 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:55.242713 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 316/634] 10.222.0.39 () {56 vars in 1112 bytes} [Fri Oct 14 02:49:55 2022] GET /identity/v3/services?name=network => generated 124 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:55.250859 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fca4a54-2bfa-4810-b926-10ea6fe8c604 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:55.333744 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:55.383818 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fca4a54-2bfa-4810-b926-10ea6fe8c604 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:55.425205 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac8c32c-fcb5-4dbd-9aca-93bca498503a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:55.426391 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac8c32c-fcb5-4dbd-9aca-93bca498503a None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:55.426676 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac8c32c-fcb5-4dbd-9aca-93bca498503a None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:55.427000 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac8c32c-fcb5-4dbd-9aca-93bca498503a None admin] PATH_INFO: `/v3/services` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:55.427642 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac8c32c-fcb5-4dbd-9aca-93bca498503a None admin] RBAC: Adding query filter params (type=network) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:55.427949 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac8c32c-fcb5-4dbd-9aca-93bca498503a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:55.429331 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac8c32c-fcb5-4dbd-9aca-93bca498503a None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:55.438076 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 319/635] 10.222.0.39 () {56 vars in 1112 bytes} [Fri Oct 14 02:49:55 2022] GET /identity/v3/services?type=network => generated 350 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:55.444081 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbd857a-435e-481e-8edf-4d9d4c594320 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:55.486597 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac8c32c-fcb5-4dbd-9aca-93bca498503a None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:55.588415 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbd857a-435e-481e-8edf-4d9d4c594320 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:55.619216 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:55.651751 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185b01bf-84b0-4c6f-9016-b4b5d156e117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:55.652758 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185b01bf-84b0-4c6f-9016-b4b5d156e117 None admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:55.653250 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185b01bf-84b0-4c6f-9016-b4b5d156e117 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:55.653743 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185b01bf-84b0-4c6f-9016-b4b5d156e117 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:55.654649 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185b01bf-84b0-4c6f-9016-b4b5d156e117 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:55.656225 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185b01bf-84b0-4c6f-9016-b4b5d156e117 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:55.670804 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac8c32c-fcb5-4dbd-9aca-93bca498503a None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:55.703776 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:55.710358 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83db8688-88e3-4f38-b3d4-245d3318f947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:55.711268 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83db8688-88e3-4f38-b3d4-245d3318f947 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:55.711754 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83db8688-88e3-4f38-b3d4-245d3318f947 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:55.712238 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83db8688-88e3-4f38-b3d4-245d3318f947 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:55.713077 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83db8688-88e3-4f38-b3d4-245d3318f947 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:55.714843 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83db8688-88e3-4f38-b3d4-245d3318f947 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:55.724694 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-83db8688-88e3-4f38-b3d4-245d3318f947 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 14 02:49:55.724694 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:55.724694 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 14 02:49:55.724694 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 14 02:49:55.724694 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 14 02:49:55.724694 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 14 02:49:55.724694 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 14 02:49:55.724694 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:49:55.724694 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 14 02:49:55.724694 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:49:55.724694 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 02:49:55.724694 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:49:55.724694 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:55.724694 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:55.724694 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:55.724694 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:55.724694 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:55.725878 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:55.725878 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:55.725878 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:55.725878 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:55.725878 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:55.725878 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:55.725878 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 14 02:49:55.725878 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 14 02:49:55.725878 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 14 02:49:55.725878 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 14 02:49:55.725878 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:55.725878 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:55.725878 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:55.725878 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:55.725878 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:55.725878 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:55.726894 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:55.726894 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:55.726894 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:55.726894 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:55.726894 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:55.726894 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:55.726894 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:55.726894 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:55.726894 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:55.726894 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:55.726894 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 14 02:49:55.726894 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:49:55.726894 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 14 02:49:55.726894 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:49:55.729533 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 320/636] 10.222.0.39 () {56 vars in 1126 bytes} [Fri Oct 14 02:49:55 2022] GET /identity/v3/services/object-store => generated 93 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:55.739248 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83db8688-88e3-4f38-b3d4-245d3318f947 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:55.768687 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 317/637] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 02:49:55 2022] POST /identity/v3/endpoints => generated 337 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:49:55.876815 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83db8688-88e3-4f38-b3d4-245d3318f947 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:55.935202 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c10171-b0fb-4d2d-8ae9-24f3a48e03a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:55.935831 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c10171-b0fb-4d2d-8ae9-24f3a48e03a4 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:55.936108 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c10171-b0fb-4d2d-8ae9-24f3a48e03a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:55.936379 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c10171-b0fb-4d2d-8ae9-24f3a48e03a4 None admin] PATH_INFO: `/v3/services` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:55.937020 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c10171-b0fb-4d2d-8ae9-24f3a48e03a4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:55.937322 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c10171-b0fb-4d2d-8ae9-24f3a48e03a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:55.938831 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c10171-b0fb-4d2d-8ae9-24f3a48e03a4 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:55.944675 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 321/638] 10.222.0.39 () {56 vars in 1122 bytes} [Fri Oct 14 02:49:55 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:55.950690 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185b01bf-84b0-4c6f-9016-b4b5d156e117 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:56.077957 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185b01bf-84b0-4c6f-9016-b4b5d156e117 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:56.116229 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49373b18-251a-4d02-80d9-a346e304a0ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:56.116913 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49373b18-251a-4d02-80d9-a346e304a0ed None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:56.117204 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49373b18-251a-4d02-80d9-a346e304a0ed None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:56.117516 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49373b18-251a-4d02-80d9-a346e304a0ed None admin] PATH_INFO: `/v3/services` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:56.118264 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49373b18-251a-4d02-80d9-a346e304a0ed None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:56.118576 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49373b18-251a-4d02-80d9-a346e304a0ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:56.120034 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49373b18-251a-4d02-80d9-a346e304a0ed None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:56.125010 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 318/639] 10.222.0.39 () {56 vars in 1122 bytes} [Fri Oct 14 02:49:55 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:56.403069 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26002d77-b958-42ec-b146-bd52d3d17f42 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:56.403614 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26002d77-b958-42ec-b146-bd52d3d17f42 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:56.403614 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26002d77-b958-42ec-b146-bd52d3d17f42 None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:56.405204 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 322/640] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:49:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:49:56.410532 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d98d2f6-9755-4039-a3c7-2fef969b97ee None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:56.410821 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d98d2f6-9755-4039-a3c7-2fef969b97ee None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:56.411096 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d98d2f6-9755-4039-a3c7-2fef969b97ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:56.429338 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-1d98d2f6-9755-4039-a3c7-2fef969b97ee None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:49:56.430580 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-1d98d2f6-9755-4039-a3c7-2fef969b97ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:49:56.465256 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:56.507018 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 319/641] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:49:56 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:49:56.774551 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26002d77-b958-42ec-b146-bd52d3d17f42 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:56.833108 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:56.885365 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26002d77-b958-42ec-b146-bd52d3d17f42 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:56.920853 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3156ea-52e7-4835-ae5f-14f519ed6d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:56.921470 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3156ea-52e7-4835-ae5f-14f519ed6d6b None admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:56.921738 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3156ea-52e7-4835-ae5f-14f519ed6d6b None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:56.922037 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3156ea-52e7-4835-ae5f-14f519ed6d6b None admin] PATH_INFO: `/v3/services` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:56.922684 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3156ea-52e7-4835-ae5f-14f519ed6d6b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:56.924044 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3156ea-52e7-4835-ae5f-14f519ed6d6b None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:56.961415 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65e6381-fced-482d-ae6d-1d3ab95f71f1 None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:56.962067 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65e6381-fced-482d-ae6d-1d3ab95f71f1 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:56.962232 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65e6381-fced-482d-ae6d-1d3ab95f71f1 None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:56.965801 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 320/642] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:49:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 14 02:49:56.968822 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2433966-8df7-4abf-947b-4b45d5f08047 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:56.969108 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2433966-8df7-4abf-947b-4b45d5f08047 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:56.969379 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2433966-8df7-4abf-947b-4b45d5f08047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:56.994136 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-a2433966-8df7-4abf-947b-4b45d5f08047 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:49:56.995873 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-a2433966-8df7-4abf-947b-4b45d5f08047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:49:56.996002 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 323/643] 10.222.0.39 () {60 vars in 1141 bytes} [Fri Oct 14 02:49:56 2022] POST /identity/v3/services => generated 269 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:49:57.093506 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 321/644] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:49:56 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 14 02:49:57.352900 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ad470-8d00-4ec9-b486-5f05f912887e None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:57.353083 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ad470-8d00-4ec9-b486-5f05f912887e None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:57.353362 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ad470-8d00-4ec9-b486-5f05f912887e None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:57.355275 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 324/645] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:49:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:49:57.360545 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0ca99e-b012-4202-95af-dc3106734d41 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:57.360836 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0ca99e-b012-4202-95af-dc3106734d41 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:57.361115 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0ca99e-b012-4202-95af-dc3106734d41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:57.362698 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420ad470-8d00-4ec9-b486-5f05f912887e None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:57.379706 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-bf0ca99e-b012-4202-95af-dc3106734d41 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:49:57.380893 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-bf0ca99e-b012-4202-95af-dc3106734d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:49:57.460948 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 322/646] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:49:57 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:49:57.481514 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420ad470-8d00-4ec9-b486-5f05f912887e None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:57.509595 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:57.517592 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da44c72a-2f47-4d8f-bd7a-6d0b820ebcc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:57.518439 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da44c72a-2f47-4d8f-bd7a-6d0b820ebcc7 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:57.518870 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da44c72a-2f47-4d8f-bd7a-6d0b820ebcc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:57.519788 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da44c72a-2f47-4d8f-bd7a-6d0b820ebcc7 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:57.520514 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da44c72a-2f47-4d8f-bd7a-6d0b820ebcc7 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:57.522183 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da44c72a-2f47-4d8f-bd7a-6d0b820ebcc7 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:57.531431 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-da44c72a-2f47-4d8f-bd7a-6d0b820ebcc7 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 14 02:49:57.531431 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:57.531431 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 14 02:49:57.531431 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 14 02:49:57.531431 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 14 02:49:57.531431 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 14 02:49:57.531431 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 14 02:49:57.531431 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:49:57.531431 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 14 02:49:57.531431 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:49:57.531431 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 02:49:57.531431 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:49:57.531431 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:57.531431 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:57.531431 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:57.531431 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:57.531431 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:57.532578 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:57.532578 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:57.532578 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:57.532578 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:57.532578 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:57.532578 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:57.532578 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 14 02:49:57.532578 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 14 02:49:57.532578 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 14 02:49:57.532578 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 14 02:49:57.532578 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:57.532578 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:57.532578 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:57.532578 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:57.532578 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:57.532578 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:57.533408 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:57.533408 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:57.533408 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:57.533408 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:57.533408 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:57.533408 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:57.533408 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:57.533408 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:57.533408 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:57.533408 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:57.533408 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 14 02:49:57.533408 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:49:57.533408 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 14 02:49:57.533408 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:49:57.533408 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 325/647] 10.222.0.39 () {56 vars in 1114 bytes} [Fri Oct 14 02:49:57 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:57.542937 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0ca99e-b012-4202-95af-dc3106734d41 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:57.567890 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:57.643585 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0ca99e-b012-4202-95af-dc3106734d41 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:57.678889 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d1bb7d-a158-4a38-b3a0-c2bfd2493782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:57.679744 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d1bb7d-a158-4a38-b3a0-c2bfd2493782 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:57.680224 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d1bb7d-a158-4a38-b3a0-c2bfd2493782 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:57.680669 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d1bb7d-a158-4a38-b3a0-c2bfd2493782 None admin] PATH_INFO: `/v3/services` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:57.681465 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d1bb7d-a158-4a38-b3a0-c2bfd2493782 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:57.681971 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d1bb7d-a158-4a38-b3a0-c2bfd2493782 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:57.683572 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d1bb7d-a158-4a38-b3a0-c2bfd2493782 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:57.693011 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 323/648] 10.222.0.39 () {56 vars in 1114 bytes} [Fri Oct 14 02:49:57 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:57.694617 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da44c72a-2f47-4d8f-bd7a-6d0b820ebcc7 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:57.749239 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d1bb7d-a158-4a38-b3a0-c2bfd2493782 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:57.803098 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da44c72a-2f47-4d8f-bd7a-6d0b820ebcc7 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:57.839835 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2463d8f0-3c4c-4eba-ba52-2eda410ccd48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:57.840581 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2463d8f0-3c4c-4eba-ba52-2eda410ccd48 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:57.840940 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2463d8f0-3c4c-4eba-ba52-2eda410ccd48 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:57.841288 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2463d8f0-3c4c-4eba-ba52-2eda410ccd48 None admin] PATH_INFO: `/v3/services` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:57.842173 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2463d8f0-3c4c-4eba-ba52-2eda410ccd48 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:57.842333 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2463d8f0-3c4c-4eba-ba52-2eda410ccd48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:57.843833 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2463d8f0-3c4c-4eba-ba52-2eda410ccd48 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:57.849795 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 326/649] 10.222.0.39 () {56 vars in 1114 bytes} [Fri Oct 14 02:49:57 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:57.873213 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:57.879108 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d1bb7d-a158-4a38-b3a0-c2bfd2493782 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:57.916172 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce051b7-c0e7-41f3-bfbb-b8ec9a5980a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:57.916745 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce051b7-c0e7-41f3-bfbb-b8ec9a5980a0 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:57.917029 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce051b7-c0e7-41f3-bfbb-b8ec9a5980a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:57.917301 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce051b7-c0e7-41f3-bfbb-b8ec9a5980a0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:57.930245 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce051b7-c0e7-41f3-bfbb-b8ec9a5980a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:49:57.936109 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-fce051b7-c0e7-41f3-bfbb-b8ec9a5980a0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 14 02:49:57.936109 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:57.936109 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:57.936109 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:57.936109 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:57.936109 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:57.936109 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:57.936109 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:57.936109 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:57.936109 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:57.936109 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:57.936109 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:57.936109 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:49:57.936109 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:49:57.936109 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:49:57.936109 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:49:57.937296 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:49:57.937296 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:49:57.937296 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:49:57.937296 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:49:57.937296 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:57.937296 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:57.937296 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:57.937296 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:57.937296 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:57.937296 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:57.937296 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:57.937296 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:57.937296 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:57.937296 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:57.937296 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:57.937296 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:57.938324 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:57.938324 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:57.938324 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:57.938324 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:57.938324 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:49:57.938324 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:49:57.938324 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:49:57.938324 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:49:57.938324 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:49:57.938324 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:49:57.938324 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 14 02:49:57.938324 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:57.941961 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 324/650] 10.222.0.39 () {56 vars in 1120 bytes} [Fri Oct 14 02:49:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 02:49:57.947753 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2463d8f0-3c4c-4eba-ba52-2eda410ccd48 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:57.999175 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:58.073521 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2463d8f0-3c4c-4eba-ba52-2eda410ccd48 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:58.108838 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b190de0-d45c-4455-aab8-74a22a7010cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:58.109789 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b190de0-d45c-4455-aab8-74a22a7010cd None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:58.110249 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b190de0-d45c-4455-aab8-74a22a7010cd None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:58.110743 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b190de0-d45c-4455-aab8-74a22a7010cd None admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:58.111574 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b190de0-d45c-4455-aab8-74a22a7010cd None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:58.112072 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b190de0-d45c-4455-aab8-74a22a7010cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:58.113792 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b190de0-d45c-4455-aab8-74a22a7010cd None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:58.124203 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 327/651] 10.222.0.39 () {56 vars in 1115 bytes} [Fri Oct 14 02:49:57 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:58.129973 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce051b7-c0e7-41f3-bfbb-b8ec9a5980a0 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:58.236676 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce051b7-c0e7-41f3-bfbb-b8ec9a5980a0 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:58.272098 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f652db5-eaa5-4c99-bb7d-4600ce07b6d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:58.272731 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f652db5-eaa5-4c99-bb7d-4600ce07b6d9 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:58.273011 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f652db5-eaa5-4c99-bb7d-4600ce07b6d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:58.273281 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f652db5-eaa5-4c99-bb7d-4600ce07b6d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:58.284687 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f652db5-eaa5-4c99-bb7d-4600ce07b6d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:58.286242 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f652db5-eaa5-4c99-bb7d-4600ce07b6d9 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:58.293936 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:58.303547 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-6f652db5-eaa5-4c99-bb7d-4600ce07b6d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:58.303547 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:58.303547 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:58.303547 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:58.303547 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:58.303547 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:58.303547 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:58.303547 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:58.303547 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:58.303547 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:58.303547 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:58.303547 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:58.303547 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:49:58.303547 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:49:58.303547 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:49:58.303547 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:49:58.304605 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:58.304605 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:58.304605 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:58.304605 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:58.304605 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:58.304605 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:58.304605 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:58.304605 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:58.304605 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:58.304605 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:58.304605 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:58.304605 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:58.304605 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:58.304605 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:58.304605 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:58.304605 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:58.304605 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:49:58.314875 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:49:58.314875 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:58.314875 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:58.314875 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 325/652] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:49:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:58.315176 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b190de0-d45c-4455-aab8-74a22a7010cd None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:58.423127 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b190de0-d45c-4455-aab8-74a22a7010cd None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:58.425479 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:58.436537 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca4a850-2f2b-4c9b-8f51-130ba7345918 None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:58.436823 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca4a850-2f2b-4c9b-8f51-130ba7345918 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:58.437100 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca4a850-2f2b-4c9b-8f51-130ba7345918 None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:58.438455 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 326/653] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:49:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:49:58.443564 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1939e5-b205-48a4-b3d0-4d79ed16fb00 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:58.446357 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1939e5-b205-48a4-b3d0-4d79ed16fb00 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:58.446357 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1939e5-b205-48a4-b3d0-4d79ed16fb00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:58.459151 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b911385a-bfd4-45b6-8e6b-bddc74c4b067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:58.459758 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b911385a-bfd4-45b6-8e6b-bddc74c4b067 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:58.460036 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b911385a-bfd4-45b6-8e6b-bddc74c4b067 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:58.460308 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b911385a-bfd4-45b6-8e6b-bddc74c4b067 None admin] PATH_INFO: `/v3/domains` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:58.461006 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b911385a-bfd4-45b6-8e6b-bddc74c4b067 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:58.461316 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b911385a-bfd4-45b6-8e6b-bddc74c4b067 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:58.462773 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b911385a-bfd4-45b6-8e6b-bddc74c4b067 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:58.463982 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-be1939e5-b205-48a4-b3d0-4d79ed16fb00 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:49:58.465997 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-be1939e5-b205-48a4-b3d0-4d79ed16fb00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:49:58.475932 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 328/654] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Oct 14 02:49:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:58.482837 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b911385a-bfd4-45b6-8e6b-bddc74c4b067 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:58.543981 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 327/655] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:49:58 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 14 02:49:58.590401 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b911385a-bfd4-45b6-8e6b-bddc74c4b067 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:58.631739 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b7bf11-cb68-43cb-adb8-72a50b8ef64a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:58.632367 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b7bf11-cb68-43cb-adb8-72a50b8ef64a None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:58.632640 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b7bf11-cb68-43cb-adb8-72a50b8ef64a None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:58.632915 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b7bf11-cb68-43cb-adb8-72a50b8ef64a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:58.640771 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b7bf11-cb68-43cb-adb8-72a50b8ef64a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:58.642261 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b7bf11-cb68-43cb-adb8-72a50b8ef64a None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:58.654615 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-c4b7bf11-cb68-43cb-adb8-72a50b8ef64a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 14 02:49:58.654615 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:58.654615 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:58.654615 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:58.654615 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:58.654615 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:58.654615 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:58.654615 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:58.654615 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:58.654615 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:58.654615 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:58.654615 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:58.654615 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 14 02:49:58.654615 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 14 02:49:58.654615 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 14 02:49:58.654615 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 14 02:49:58.655684 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:58.655684 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:58.655684 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 14 02:49:58.655684 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:49:58.655684 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 14 02:49:58.655684 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:49:58.655684 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:58.655684 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:58.655684 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:58.655684 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:58.655684 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:58.655684 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:58.655684 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:58.655684 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:58.655684 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:58.655684 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:58.655684 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:58.655684 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:58.656632 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:58.656632 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:58.656632 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 14 02:49:58.656632 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 14 02:49:58.656632 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 14 02:49:58.656632 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 14 02:49:58.656632 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 14 02:49:58.656632 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 14 02:49:58.656632 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 14 02:49:58.656632 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:49:58.656632 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 329/656] 10.222.0.39 () {56 vars in 1117 bytes} [Fri Oct 14 02:49:58 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:58.663514 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1939e5-b205-48a4-b3d0-4d79ed16fb00 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:58.766454 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1939e5-b205-48a4-b3d0-4d79ed16fb00 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:58.798258 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:58.806418 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e00a6d-7703-4dd4-af61-46c17c8e758e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:58.807048 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e00a6d-7703-4dd4-af61-46c17c8e758e None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:58.807389 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e00a6d-7703-4dd4-af61-46c17c8e758e None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:58.807703 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e00a6d-7703-4dd4-af61-46c17c8e758e None admin] PATH_INFO: `/v3/users` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:58.808536 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e00a6d-7703-4dd4-af61-46c17c8e758e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:58.808894 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e00a6d-7703-4dd4-af61-46c17c8e758e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:58.810530 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e00a6d-7703-4dd4-af61-46c17c8e758e None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:58.831069 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 328/657] 10.222.0.39 () {56 vars in 1149 bytes} [Fri Oct 14 02:49:58 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:58.837455 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b7bf11-cb68-43cb-adb8-72a50b8ef64a None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:58.865293 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:58.868121 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e00a6d-7703-4dd4-af61-46c17c8e758e None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:58.944106 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b7bf11-cb68-43cb-adb8-72a50b8ef64a None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:58.979960 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f301dc5c-53a3-4a7c-b7d9-3db2137fe46b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:58.981025 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f301dc5c-53a3-4a7c-b7d9-3db2137fe46b None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:58.981514 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f301dc5c-53a3-4a7c-b7d9-3db2137fe46b None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:58.982029 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f301dc5c-53a3-4a7c-b7d9-3db2137fe46b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:58.982157 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e00a6d-7703-4dd4-af61-46c17c8e758e None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:58.993072 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f301dc5c-53a3-4a7c-b7d9-3db2137fe46b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:58.994806 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f301dc5c-53a3-4a7c-b7d9-3db2137fe46b None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:59.015876 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-f301dc5c-53a3-4a7c-b7d9-3db2137fe46b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:59.015876 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:59.015876 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:59.015876 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:59.015876 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:59.015876 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:59.015876 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:59.015876 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:59.015876 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:59.015876 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:59.015876 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:59.015876 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:59.015876 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:49:59.015876 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:49:59.015876 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:49:59.015876 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:49:59.016880 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:59.016880 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:59.016880 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:59.016880 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:59.016880 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:59.016880 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:59.016880 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:59.016880 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:59.016880 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:59.016880 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:59.016880 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:59.016880 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:59.016880 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:59.016880 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:59.016880 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:59.016880 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:59.016880 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:49:59.018212 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:49:59.018212 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:49:59.018212 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:49:59.021700 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 330/658] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:49:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:59.022542 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f301dc5c-53a3-4a7c-b7d9-3db2137fe46b None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:59.024252 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d03811-983d-4c33-87d2-7464fd231515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:59.025313 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d03811-983d-4c33-87d2-7464fd231515 None admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:59.025852 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d03811-983d-4c33-87d2-7464fd231515 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:59.026360 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d03811-983d-4c33-87d2-7464fd231515 None admin] PATH_INFO: `/v3/services` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:59.027236 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d03811-983d-4c33-87d2-7464fd231515 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:59.028829 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d03811-983d-4c33-87d2-7464fd231515 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:59.101414 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 329/659] 10.222.0.39 () {60 vars in 1141 bytes} [Fri Oct 14 02:49:58 2022] POST /identity/v3/services => generated 241 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 02:49:59.113308 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:59.146230 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f301dc5c-53a3-4a7c-b7d9-3db2137fe46b None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:59.195904 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a191d2c-c75a-42b0-a8a5-3a7dfa3e780e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:59.196907 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a191d2c-c75a-42b0-a8a5-3a7dfa3e780e None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:59.197375 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a191d2c-c75a-42b0-a8a5-3a7dfa3e780e None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:59.197894 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a191d2c-c75a-42b0-a8a5-3a7dfa3e780e None admin] PATH_INFO: `/v3/domains` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:59.198805 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a191d2c-c75a-42b0-a8a5-3a7dfa3e780e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:59.199300 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a191d2c-c75a-42b0-a8a5-3a7dfa3e780e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:59.200979 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a191d2c-c75a-42b0-a8a5-3a7dfa3e780e None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:59.220086 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 331/660] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Oct 14 02:49:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:59.224673 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d03811-983d-4c33-87d2-7464fd231515 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:59.249635 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a46514-9709-4380-97af-c748a79e948e None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:59.249635 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a46514-9709-4380-97af-c748a79e948e None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:59.249967 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a46514-9709-4380-97af-c748a79e948e None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:59.251875 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 332/661] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:49:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 14 02:49:59.258244 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086130a4-22a0-47cd-bb3f-11fb359bd066 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:59.258659 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086130a4-22a0-47cd-bb3f-11fb359bd066 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:59.259025 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086130a4-22a0-47cd-bb3f-11fb359bd066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:59.281147 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-086130a4-22a0-47cd-bb3f-11fb359bd066 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:49:59.283162 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-086130a4-22a0-47cd-bb3f-11fb359bd066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:49:59.285439 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:59.350994 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d03811-983d-4c33-87d2-7464fd231515 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:59.367670 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 333/662] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:49:59 2022] POST /identity/v3/auth/tokens => generated 1373 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 14 02:49:59.388452 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459aea41-37a5-4ea6-8080-0c4deba19e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:59.389197 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459aea41-37a5-4ea6-8080-0c4deba19e8f None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:59.389555 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459aea41-37a5-4ea6-8080-0c4deba19e8f None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:59.389924 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459aea41-37a5-4ea6-8080-0c4deba19e8f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:59.396875 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459aea41-37a5-4ea6-8080-0c4deba19e8f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:59.398368 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459aea41-37a5-4ea6-8080-0c4deba19e8f None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:59.410223 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-459aea41-37a5-4ea6-8080-0c4deba19e8f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:49:59.410223 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:59.410223 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:59.410223 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:59.410223 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:59.410223 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:59.410223 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:59.410223 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:59.410223 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:59.410223 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:59.410223 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:59.410223 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:59.410223 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 14 02:49:59.410223 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 14 02:49:59.410223 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 14 02:49:59.411113 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 14 02:49:59.411113 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:59.411113 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:59.411113 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:59.411113 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:59.411113 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:59.411113 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:59.411113 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:59.411113 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:59.411113 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:59.411113 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:59.411113 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:59.411113 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:59.411113 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:59.411113 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:59.411113 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:59.411113 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:59.411991 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 14 02:49:59.411991 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 14 02:49:59.411991 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 14 02:49:59.411991 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 14 02:49:59.411991 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 14 02:49:59.411991 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 14 02:49:59.411991 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:49:59.411991 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:49:59.411991 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 330/663] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Oct 14 02:49:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:59.418692 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086130a4-22a0-47cd-bb3f-11fb359bd066 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:59.482378 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:59.522733 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086130a4-22a0-47cd-bb3f-11fb359bd066 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:59.559709 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9302b426-b12c-4103-95c2-8a399bb780fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:59.560324 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9302b426-b12c-4103-95c2-8a399bb780fb None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:59.560610 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9302b426-b12c-4103-95c2-8a399bb780fb None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:59.560886 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9302b426-b12c-4103-95c2-8a399bb780fb None admin] PATH_INFO: `/v3/projects` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:59.561586 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9302b426-b12c-4103-95c2-8a399bb780fb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:59.561926 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9302b426-b12c-4103-95c2-8a399bb780fb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:59.563396 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9302b426-b12c-4103-95c2-8a399bb780fb None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:59.575611 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 334/664] 10.222.0.39 () {56 vars in 1148 bytes} [Fri Oct 14 02:49:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:59.581680 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459aea41-37a5-4ea6-8080-0c4deba19e8f None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:59.636295 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9302b426-b12c-4103-95c2-8a399bb780fb None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:59.700318 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459aea41-37a5-4ea6-8080-0c4deba19e8f None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:59.709968 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:59.746193 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70d54f2-bcb3-4e1d-9dd0-dec9218c9e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:59.746855 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d54f2-bcb3-4e1d-9dd0-dec9218c9e12 None admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:59.747133 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d54f2-bcb3-4e1d-9dd0-dec9218c9e12 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:59.747406 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d54f2-bcb3-4e1d-9dd0-dec9218c9e12 None admin] PATH_INFO: `/v3/projects/dac9f8cd285c44e889a621b8b7fe479e/users/44b37050076b42e6af1a67bd7d6d06e1/roles/22dff4ef49264f709f154e98d331d850` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:59.749662 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70d54f2-bcb3-4e1d-9dd0-dec9218c9e12 None admin] RBAC: Authorizing `identity:create_grant(project_id=dac9f8cd285c44e889a621b8b7fe479e, user_id=44b37050076b42e6af1a67bd7d6d06e1, role_id=22dff4ef49264f709f154e98d331d850)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:59.751119 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70d54f2-bcb3-4e1d-9dd0-dec9218c9e12 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:59.756125 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9302b426-b12c-4103-95c2-8a399bb780fb None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:59.780241 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:49:59.792683 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98cbbed-bd02-48c7-b528-584a5360dd64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:59.793438 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98cbbed-bd02-48c7-b528-584a5360dd64 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:59.793718 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98cbbed-bd02-48c7-b528-584a5360dd64 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:59.794026 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98cbbed-bd02-48c7-b528-584a5360dd64 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:59.794617 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98cbbed-bd02-48c7-b528-584a5360dd64 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:59.795996 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98cbbed-bd02-48c7-b528-584a5360dd64 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:59.804701 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-d98cbbed-bd02-48c7-b528-584a5360dd64 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 14 02:49:59.804701 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:59.804701 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 14 02:49:59.804701 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 14 02:49:59.804701 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 14 02:49:59.804701 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 14 02:49:59.804701 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 14 02:49:59.804701 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:49:59.804701 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 14 02:49:59.804701 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:49:59.804701 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 02:49:59.804701 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:49:59.804701 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:49:59.804701 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:49:59.804701 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:49:59.804701 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:49:59.809978 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:49:59.809978 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:49:59.809978 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:49:59.809978 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:49:59.809978 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:49:59.809978 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:49:59.809978 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:49:59.809978 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 14 02:49:59.809978 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 14 02:49:59.809978 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 14 02:49:59.809978 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 14 02:49:59.809978 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:49:59.809978 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:49:59.809978 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:49:59.809978 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:49:59.809978 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:49:59.809978 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:49:59.811041 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:49:59.811041 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:49:59.811041 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:49:59.811041 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:49:59.811041 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:49:59.811041 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:49:59.811041 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:49:59.811041 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:49:59.811041 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:49:59.811041 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:49:59.811041 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 14 02:49:59.811041 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:49:59.811041 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 14 02:49:59.811041 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:49:59.811041 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 335/665] 10.222.0.39 () {56 vars in 1132 bytes} [Fri Oct 14 02:49:59 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:49:59.812819 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98cbbed-bd02-48c7-b528-584a5360dd64 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:49:59.827150 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 331/666] 10.222.0.39 () {58 vars in 1439 bytes} [Fri Oct 14 02:49:59 2022] PUT /identity/v3/projects/dac9f8cd285c44e889a621b8b7fe479e/users/44b37050076b42e6af1a67bd7d6d06e1/roles/22dff4ef49264f709f154e98d331d850 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:49:59.936096 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98cbbed-bd02-48c7-b528-584a5360dd64 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:49:59.976351 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d91f3c-4654-4ca3-b07d-b3b8e2c2cef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:49:59.977401 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d91f3c-4654-4ca3-b07d-b3b8e2c2cef9 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:49:59.977916 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d91f3c-4654-4ca3-b07d-b3b8e2c2cef9 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:49:59.978414 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d91f3c-4654-4ca3-b07d-b3b8e2c2cef9 None admin] PATH_INFO: `/v3/services` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:49:59.979531 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d91f3c-4654-4ca3-b07d-b3b8e2c2cef9 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:49:59.980162 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d91f3c-4654-4ca3-b07d-b3b8e2c2cef9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:49:59.981987 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d91f3c-4654-4ca3-b07d-b3b8e2c2cef9 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:49:59.988268 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 336/667] 10.222.0.39 () {56 vars in 1126 bytes} [Fri Oct 14 02:49:59 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:49:59.994737 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70d54f2-bcb3-4e1d-9dd0-dec9218c9e12 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:00.109145 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70d54f2-bcb3-4e1d-9dd0-dec9218c9e12 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:00.126248 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af18b2-ccfb-495b-b73e-6567d442dede None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:00.126653 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af18b2-ccfb-495b-b73e-6567d442dede None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:00.126995 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af18b2-ccfb-495b-b73e-6567d442dede None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:00.129385 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 337/668] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:50:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:50:00.137408 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81251d3e-f5a3-4692-bdaf-4a8b66a05a5e None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:00.137934 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81251d3e-f5a3-4692-bdaf-4a8b66a05a5e None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:00.138394 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81251d3e-f5a3-4692-bdaf-4a8b66a05a5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:00.149613 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b572ff3-68a3-4ac5-8013-2806065fb206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:00.150417 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b572ff3-68a3-4ac5-8013-2806065fb206 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:00.150845 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b572ff3-68a3-4ac5-8013-2806065fb206 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:00.151222 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b572ff3-68a3-4ac5-8013-2806065fb206 None admin] PATH_INFO: `/v3/services` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:00.151962 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b572ff3-68a3-4ac5-8013-2806065fb206 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:50:00.152368 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b572ff3-68a3-4ac5-8013-2806065fb206 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:00.154633 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b572ff3-68a3-4ac5-8013-2806065fb206 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:00.158467 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-81251d3e-f5a3-4692-bdaf-4a8b66a05a5e None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:50:00.159676 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-81251d3e-f5a3-4692-bdaf-4a8b66a05a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:50:00.207874 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:00.232341 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 332/670] 10.222.0.39 () {56 vars in 1126 bytes} [Fri Oct 14 02:49:59 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:50:00.232839 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 338/670] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:50:00 2022] POST /identity/v3/auth/tokens => generated 1373 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:50:00.239190 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81251d3e-f5a3-4692-bdaf-4a8b66a05a5e None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:00.344797 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81251d3e-f5a3-4692-bdaf-4a8b66a05a5e None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:00.380394 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80f00e5-6f30-4fa0-951d-5c917b1dc104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:00.381006 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80f00e5-6f30-4fa0-951d-5c917b1dc104 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:00.381294 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80f00e5-6f30-4fa0-951d-5c917b1dc104 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:00.381566 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80f00e5-6f30-4fa0-951d-5c917b1dc104 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:00.382368 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80f00e5-6f30-4fa0-951d-5c917b1dc104 None admin] RBAC: Adding query filter params (interface=public, service_id=725d99934ad34ce487b4724fa44b0d48, region_id=RegionOne) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:50:00.382684 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80f00e5-6f30-4fa0-951d-5c917b1dc104 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:00.384134 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80f00e5-6f30-4fa0-951d-5c917b1dc104 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:00.389591 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 339/671] 10.222.0.39 () {56 vars in 1251 bytes} [Fri Oct 14 02:50:00 2022] GET /identity/v3/endpoints?service_id=725d99934ad34ce487b4724fa44b0d48&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:50:00.395864 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b572ff3-68a3-4ac5-8013-2806065fb206 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:00.428524 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:00.504884 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b572ff3-68a3-4ac5-8013-2806065fb206 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:00.540098 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5c4464-6b73-4ad1-a371-3cf189d19dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:00.540943 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5c4464-6b73-4ad1-a371-3cf189d19dae None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:00.541377 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5c4464-6b73-4ad1-a371-3cf189d19dae None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:00.541875 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5c4464-6b73-4ad1-a371-3cf189d19dae None admin] PATH_INFO: `/v3/services` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:00.542628 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5c4464-6b73-4ad1-a371-3cf189d19dae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:00.544138 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5c4464-6b73-4ad1-a371-3cf189d19dae None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:00.547911 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80f00e5-6f30-4fa0-951d-5c917b1dc104 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:00.552099 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 333/672] 10.222.0.39 () {56 vars in 1087 bytes} [Fri Oct 14 02:50:00 2022] GET /identity/v3/services => generated 1258 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:50:00.604133 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:00.668901 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80f00e5-6f30-4fa0-951d-5c917b1dc104 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:00.708161 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3121ad-0888-4405-9cd6-cd50919e0d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:00.708943 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3121ad-0888-4405-9cd6-cd50919e0d7a None admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:00.709329 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3121ad-0888-4405-9cd6-cd50919e0d7a None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:00.709707 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3121ad-0888-4405-9cd6-cd50919e0d7a None admin] PATH_INFO: `/v3/services` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:00.710517 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3121ad-0888-4405-9cd6-cd50919e0d7a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:00.712048 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3121ad-0888-4405-9cd6-cd50919e0d7a None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:00.779955 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 340/673] 10.222.0.39 () {60 vars in 1141 bytes} [Fri Oct 14 02:50:00 2022] POST /identity/v3/services => generated 251 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:50:01.370359 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c677c42c-cc23-499b-99cb-59c7756f3321 None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:01.371425 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c677c42c-cc23-499b-99cb-59c7756f3321 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:01.371920 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c677c42c-cc23-499b-99cb-59c7756f3321 None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:01.373801 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 334/674] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:50:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:50:01.379525 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e216d8-4502-4f9a-bbaf-0ee402a8e012 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:01.379816 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e216d8-4502-4f9a-bbaf-0ee402a8e012 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:01.380089 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e216d8-4502-4f9a-bbaf-0ee402a8e012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:01.398469 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-37e216d8-4502-4f9a-bbaf-0ee402a8e012 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:50:01.399713 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-37e216d8-4502-4f9a-bbaf-0ee402a8e012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:50:01.472717 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 341/675] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:50:01 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 14 02:50:01.745930 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c677c42c-cc23-499b-99cb-59c7756f3321 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:01.843549 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:01.852771 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c677c42c-cc23-499b-99cb-59c7756f3321 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:01.888645 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a641f4-b351-4119-b5c4-a68b6a1f974e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:01.889651 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a641f4-b351-4119-b5c4-a68b6a1f974e None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:01.890176 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a641f4-b351-4119-b5c4-a68b6a1f974e None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:01.891740 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a641f4-b351-4119-b5c4-a68b6a1f974e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:01.892564 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a641f4-b351-4119-b5c4-a68b6a1f974e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:01.894262 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a641f4-b351-4119-b5c4-a68b6a1f974e None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:01.903726 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-e8a641f4-b351-4119-b5c4-a68b6a1f974e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 14 02:50:01.903726 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:50:01.903726 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 14 02:50:01.903726 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 14 02:50:01.903726 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 14 02:50:01.903726 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 14 02:50:01.903726 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 14 02:50:01.903726 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:50:01.903726 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 14 02:50:01.903726 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:50:01.903726 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 02:50:01.903726 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:50:01.903726 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:50:01.903726 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:50:01.903726 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:50:01.903726 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:50:01.903726 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:50:01.904726 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:50:01.904726 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:50:01.904726 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:50:01.904726 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:50:01.904726 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:50:01.904726 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:50:01.904726 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 14 02:50:01.904726 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 14 02:50:01.904726 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 14 02:50:01.904726 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 14 02:50:01.904726 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:50:01.904726 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:50:01.904726 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:50:01.904726 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:50:01.904726 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:50:01.904726 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:50:01.905550 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:50:01.905550 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:50:01.905550 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:50:01.905550 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:50:01.905550 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:50:01.905550 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:50:01.905550 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:50:01.905550 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:50:01.905550 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:50:01.905550 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:50:01.905550 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 14 02:50:01.905550 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:50:01.905550 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 14 02:50:01.905550 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:50:01.907810 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 335/676] 10.222.0.39 () {56 vars in 1126 bytes} [Fri Oct 14 02:50:01 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:50:01.914026 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e216d8-4502-4f9a-bbaf-0ee402a8e012 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:01.946880 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:02.024184 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e216d8-4502-4f9a-bbaf-0ee402a8e012 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:02.061670 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ffdd8f-5114-47d7-9e0f-4d2aea73d541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:02.062322 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ffdd8f-5114-47d7-9e0f-4d2aea73d541 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:02.062605 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ffdd8f-5114-47d7-9e0f-4d2aea73d541 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:02.062882 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ffdd8f-5114-47d7-9e0f-4d2aea73d541 None admin] PATH_INFO: `/v3/services` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:02.063520 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ffdd8f-5114-47d7-9e0f-4d2aea73d541 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:50:02.063826 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ffdd8f-5114-47d7-9e0f-4d2aea73d541 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:02.065208 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ffdd8f-5114-47d7-9e0f-4d2aea73d541 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:02.070626 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 342/677] 10.222.0.39 () {56 vars in 1122 bytes} [Fri Oct 14 02:50:01 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:50:02.076113 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a641f4-b351-4119-b5c4-a68b6a1f974e None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:02.079226 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb92904-91e1-4fe3-87d8-0f80f61a1acf None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:02.079518 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb92904-91e1-4fe3-87d8-0f80f61a1acf None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:02.079792 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb92904-91e1-4fe3-87d8-0f80f61a1acf None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:02.082007 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 343/678] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:50:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:50:02.089891 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605cc156-1767-4630-9ff0-b84b822251cd None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:02.090180 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605cc156-1767-4630-9ff0-b84b822251cd None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:02.090519 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605cc156-1767-4630-9ff0-b84b822251cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:02.109970 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-605cc156-1767-4630-9ff0-b84b822251cd None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:50:02.111441 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-605cc156-1767-4630-9ff0-b84b822251cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:50:02.185153 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a641f4-b351-4119-b5c4-a68b6a1f974e None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:02.201133 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 344/679] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:50:02 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:50:02.223142 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269297a9-4056-4bb6-918a-a80a6d491fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:02.223905 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269297a9-4056-4bb6-918a-a80a6d491fcc None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:02.224279 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269297a9-4056-4bb6-918a-a80a6d491fcc None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:02.224699 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269297a9-4056-4bb6-918a-a80a6d491fcc None admin] PATH_INFO: `/v3/services` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:02.225419 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269297a9-4056-4bb6-918a-a80a6d491fcc None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:50:02.225850 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269297a9-4056-4bb6-918a-a80a6d491fcc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:02.227348 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269297a9-4056-4bb6-918a-a80a6d491fcc None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:02.232980 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 336/680] 10.222.0.39 () {56 vars in 1122 bytes} [Fri Oct 14 02:50:02 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:50:02.238575 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605cc156-1767-4630-9ff0-b84b822251cd None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:02.263789 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:02.340533 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605cc156-1767-4630-9ff0-b84b822251cd None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:02.375416 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25ad5c0-ce28-4db1-a05c-09c5f673d34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:02.376016 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ad5c0-ce28-4db1-a05c-09c5f673d34d None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:02.376346 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ad5c0-ce28-4db1-a05c-09c5f673d34d None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:02.376626 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ad5c0-ce28-4db1-a05c-09c5f673d34d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:02.377324 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25ad5c0-ce28-4db1-a05c-09c5f673d34d None admin] RBAC: Adding query filter params (interface=public, service_id=fc656af825124aca9eed0a471c2f9581, region_id=RegionOne) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:50:02.377632 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25ad5c0-ce28-4db1-a05c-09c5f673d34d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:02.379556 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25ad5c0-ce28-4db1-a05c-09c5f673d34d None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:02.385033 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 345/681] 10.222.0.39 () {56 vars in 1251 bytes} [Fri Oct 14 02:50:02 2022] GET /identity/v3/endpoints?service_id=fc656af825124aca9eed0a471c2f9581&interface=public®ion_id=RegionOne => generated 194 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:50:02.390734 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269297a9-4056-4bb6-918a-a80a6d491fcc None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:02.416178 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:02.492766 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25ad5c0-ce28-4db1-a05c-09c5f673d34d None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:02.492963 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269297a9-4056-4bb6-918a-a80a6d491fcc None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:02.532712 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564287ef-58d2-4b41-9d5d-c127f86092c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:02.533376 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564287ef-58d2-4b41-9d5d-c127f86092c4 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:02.533631 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564287ef-58d2-4b41-9d5d-c127f86092c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:02.533937 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564287ef-58d2-4b41-9d5d-c127f86092c4 None admin] PATH_INFO: `/v3/services` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:02.534704 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564287ef-58d2-4b41-9d5d-c127f86092c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:02.536061 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564287ef-58d2-4b41-9d5d-c127f86092c4 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:02.542856 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 337/682] 10.222.0.39 () {56 vars in 1087 bytes} [Fri Oct 14 02:50:02 2022] GET /identity/v3/services => generated 1497 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:50:02.604654 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:02.617916 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25ad5c0-ce28-4db1-a05c-09c5f673d34d None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:02.657637 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de542c68-0bae-49d3-bcdc-150da307bae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:02.658626 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de542c68-0bae-49d3-bcdc-150da307bae1 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:02.659096 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de542c68-0bae-49d3-bcdc-150da307bae1 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:02.659858 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de542c68-0bae-49d3-bcdc-150da307bae1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:02.671491 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de542c68-0bae-49d3-bcdc-150da307bae1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:50:02.676524 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-de542c68-0bae-49d3-bcdc-150da307bae1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 14 02:50:02.676524 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:50:02.676524 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:50:02.676524 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:50:02.676524 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:50:02.676524 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:50:02.676524 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:50:02.676524 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:50:02.676524 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:50:02.676524 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:50:02.676524 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:50:02.676524 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:50:02.676524 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:50:02.676524 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:50:02.676524 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:50:02.676524 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:50:02.677529 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:50:02.677529 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:50:02.677529 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:50:02.677529 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:50:02.677529 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:50:02.677529 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:50:02.677529 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:50:02.677529 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:50:02.677529 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:50:02.677529 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:50:02.677529 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:50:02.677529 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:50:02.677529 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:50:02.677529 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:50:02.677529 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:50:02.677529 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:50:02.678433 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:50:02.678433 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:50:02.678433 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:50:02.678433 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:50:02.678433 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:50:02.678433 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:50:02.678433 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:50:02.678433 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:50:02.678433 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:50:02.678433 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:50:02.678433 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 14 02:50:02.678433 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:50:02.678433 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 346/683] 10.222.0.39 () {56 vars in 1120 bytes} [Fri Oct 14 02:50:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 02:50:02.684031 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564287ef-58d2-4b41-9d5d-c127f86092c4 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:02.787588 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:02.802376 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564287ef-58d2-4b41-9d5d-c127f86092c4 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:02.837744 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132795a7-91b4-456d-b5a3-8d3c6af5cb62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:02.838432 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132795a7-91b4-456d-b5a3-8d3c6af5cb62 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:02.838711 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132795a7-91b4-456d-b5a3-8d3c6af5cb62 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:02.838982 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132795a7-91b4-456d-b5a3-8d3c6af5cb62 None admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:02.839638 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132795a7-91b4-456d-b5a3-8d3c6af5cb62 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:50:02.839941 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132795a7-91b4-456d-b5a3-8d3c6af5cb62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:02.841319 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132795a7-91b4-456d-b5a3-8d3c6af5cb62 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:02.854062 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 338/684] 10.222.0.39 () {56 vars in 1115 bytes} [Fri Oct 14 02:50:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:50:02.856180 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de542c68-0bae-49d3-bcdc-150da307bae1 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:02.904397 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf662d26-bba5-4986-ad7d-420fe74bcbc9 None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:02.904689 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf662d26-bba5-4986-ad7d-420fe74bcbc9 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:02.904973 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf662d26-bba5-4986-ad7d-420fe74bcbc9 None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:02.906613 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 339/685] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:50:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:50:02.911749 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc2eb39-f993-4f66-8f65-c99efe8e58fe None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:02.912042 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc2eb39-f993-4f66-8f65-c99efe8e58fe None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:02.912325 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc2eb39-f993-4f66-8f65-c99efe8e58fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:02.931555 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-3cc2eb39-f993-4f66-8f65-c99efe8e58fe None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:50:02.933122 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-3cc2eb39-f993-4f66-8f65-c99efe8e58fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:50:02.972085 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de542c68-0bae-49d3-bcdc-150da307bae1 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:03.021827 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 340/686] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:50:02 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:50:03.033838 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a95d0c7-7bab-4cd5-8f7f-578fb05d50d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:03.034755 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a95d0c7-7bab-4cd5-8f7f-578fb05d50d8 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:03.035252 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a95d0c7-7bab-4cd5-8f7f-578fb05d50d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:03.035721 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a95d0c7-7bab-4cd5-8f7f-578fb05d50d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:03.046523 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a95d0c7-7bab-4cd5-8f7f-578fb05d50d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:03.048181 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a95d0c7-7bab-4cd5-8f7f-578fb05d50d8 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:03.057570 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:03.064655 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-4a95d0c7-7bab-4cd5-8f7f-578fb05d50d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:50:03.064655 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:50:03.064655 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:50:03.064655 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:50:03.064655 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:50:03.064655 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:50:03.064655 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:50:03.064655 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:50:03.064655 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:50:03.064655 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:50:03.064655 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:50:03.064655 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:50:03.064655 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:50:03.064655 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:50:03.064655 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:50:03.064655 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:50:03.074606 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:50:03.074606 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:50:03.074606 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:50:03.074606 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:50:03.074606 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:50:03.074606 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:50:03.074606 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:50:03.074606 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:50:03.074606 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:50:03.074606 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:50:03.074606 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:50:03.074606 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:50:03.074606 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:50:03.074606 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:50:03.074606 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:50:03.074606 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:50:03.074606 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:50:03.075488 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:50:03.075488 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:50:03.075488 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:50:03.075488 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 347/687] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:50:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:50:03.075708 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc2eb39-f993-4f66-8f65-c99efe8e58fe None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:03.099182 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051530b0-2590-4ddc-a919-b3b9b8c69e61 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:03.099465 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051530b0-2590-4ddc-a919-b3b9b8c69e61 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:03.099742 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051530b0-2590-4ddc-a919-b3b9b8c69e61 None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:03.101420 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 348/688] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:50:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 14 02:50:03.106651 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889d63e2-b7c4-41f1-ac6b-d76aa594f78f None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:03.107042 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889d63e2-b7c4-41f1-ac6b-d76aa594f78f None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:03.107418 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889d63e2-b7c4-41f1-ac6b-d76aa594f78f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:03.127913 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-889d63e2-b7c4-41f1-ac6b-d76aa594f78f None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:50:03.129896 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-889d63e2-b7c4-41f1-ac6b-d76aa594f78f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:50:03.169250 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:03.196171 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc2eb39-f993-4f66-8f65-c99efe8e58fe None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:03.208929 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 349/689] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:50:03 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:50:03.236008 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefafe71-388e-4bd6-a58f-d281024f9545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:03.236615 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefafe71-388e-4bd6-a58f-d281024f9545 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:03.236891 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefafe71-388e-4bd6-a58f-d281024f9545 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:03.237162 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefafe71-388e-4bd6-a58f-d281024f9545 None admin] PATH_INFO: `/v3/domains` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:03.238431 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefafe71-388e-4bd6-a58f-d281024f9545 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:50:03.239015 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefafe71-388e-4bd6-a58f-d281024f9545 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:03.240444 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefafe71-388e-4bd6-a58f-d281024f9545 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:03.251638 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 341/690] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Oct 14 02:50:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:50:03.257321 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889d63e2-b7c4-41f1-ac6b-d76aa594f78f None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:03.291376 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefafe71-388e-4bd6-a58f-d281024f9545 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:03.369202 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889d63e2-b7c4-41f1-ac6b-d76aa594f78f None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:03.400525 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:03.407396 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f039ddf-5ed8-4def-9c46-a7413aa88e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:03.408142 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f039ddf-5ed8-4def-9c46-a7413aa88e92 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:03.408522 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f039ddf-5ed8-4def-9c46-a7413aa88e92 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:03.408889 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f039ddf-5ed8-4def-9c46-a7413aa88e92 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:03.411204 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefafe71-388e-4bd6-a58f-d281024f9545 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:03.417575 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f039ddf-5ed8-4def-9c46-a7413aa88e92 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:03.419161 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f039ddf-5ed8-4def-9c46-a7413aa88e92 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:03.432550 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-7f039ddf-5ed8-4def-9c46-a7413aa88e92 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 14 02:50:03.432550 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:50:03.432550 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:50:03.432550 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:50:03.432550 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:50:03.432550 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:50:03.432550 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:50:03.432550 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:50:03.432550 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:50:03.432550 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:50:03.432550 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:50:03.432550 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:50:03.432550 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 14 02:50:03.432550 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 14 02:50:03.432550 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 14 02:50:03.432550 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 14 02:50:03.433740 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:50:03.433740 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:50:03.433740 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 14 02:50:03.433740 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:50:03.433740 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 14 02:50:03.433740 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:50:03.433740 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:50:03.433740 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:50:03.433740 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:50:03.433740 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:50:03.433740 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:50:03.433740 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:50:03.433740 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:50:03.433740 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:50:03.433740 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:50:03.433740 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:50:03.433740 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:50:03.433740 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:50:03.434722 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:50:03.434722 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:50:03.434722 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 14 02:50:03.434722 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 14 02:50:03.434722 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 14 02:50:03.434722 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 14 02:50:03.434722 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 14 02:50:03.434722 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 14 02:50:03.434722 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 14 02:50:03.434722 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:50:03.439510 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 350/691] 10.222.0.39 () {56 vars in 1117 bytes} [Fri Oct 14 02:50:03 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:50:03.440833 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f039ddf-5ed8-4def-9c46-a7413aa88e92 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:03.454083 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69efb4df-ac4b-4522-9675-9a533fb7d60b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:03.455966 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69efb4df-ac4b-4522-9675-9a533fb7d60b None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:03.456280 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69efb4df-ac4b-4522-9675-9a533fb7d60b None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:03.456551 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69efb4df-ac4b-4522-9675-9a533fb7d60b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:03.457138 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69efb4df-ac4b-4522-9675-9a533fb7d60b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:03.458528 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69efb4df-ac4b-4522-9675-9a533fb7d60b None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:03.468503 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-69efb4df-ac4b-4522-9675-9a533fb7d60b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 14 02:50:03.468503 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:50:03.468503 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 14 02:50:03.468503 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 14 02:50:03.468503 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 14 02:50:03.468503 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 14 02:50:03.468503 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 14 02:50:03.468503 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:50:03.468503 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 14 02:50:03.468503 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:50:03.468503 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 02:50:03.468503 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:50:03.468503 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:50:03.468503 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:50:03.468503 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:50:03.468503 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:50:03.469449 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:50:03.469449 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:50:03.469449 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:50:03.469449 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:50:03.469449 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:50:03.469449 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:50:03.469449 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:50:03.469449 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 14 02:50:03.469449 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 14 02:50:03.469449 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 14 02:50:03.469449 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 14 02:50:03.469449 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:50:03.469449 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:50:03.469449 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:50:03.469449 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:50:03.469449 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:50:03.469449 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:50:03.476087 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:50:03.476087 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:50:03.476087 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:50:03.476087 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:50:03.476087 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:50:03.476087 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:50:03.476087 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:50:03.476087 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:50:03.476087 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:50:03.476087 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:50:03.476087 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 14 02:50:03.476087 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:50:03.476087 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 14 02:50:03.476087 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:50:03.476087 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 342/692] 10.222.0.39 () {56 vars in 1132 bytes} [Fri Oct 14 02:50:03 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 02:50:03.478695 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69efb4df-ac4b-4522-9675-9a533fb7d60b None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:03.483546 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:03.562115 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f039ddf-5ed8-4def-9c46-a7413aa88e92 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:03.590553 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69efb4df-ac4b-4522-9675-9a533fb7d60b None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:03.601032 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a21957a-afd7-414f-98f1-f68b215febab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:03.601970 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a21957a-afd7-414f-98f1-f68b215febab None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:03.602260 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a21957a-afd7-414f-98f1-f68b215febab None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:03.602698 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a21957a-afd7-414f-98f1-f68b215febab None admin] PATH_INFO: `/v3/users` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:03.603416 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a21957a-afd7-414f-98f1-f68b215febab None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:50:03.603730 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a21957a-afd7-414f-98f1-f68b215febab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:03.605426 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a21957a-afd7-414f-98f1-f68b215febab None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:03.622342 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 351/693] 10.222.0.39 () {56 vars in 1149 bytes} [Fri Oct 14 02:50:03 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:50:03.623073 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a21957a-afd7-414f-98f1-f68b215febab None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:03.631206 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34913513-9655-40bb-b74b-f20aa83ee8fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:03.632035 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34913513-9655-40bb-b74b-f20aa83ee8fb None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:03.632490 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34913513-9655-40bb-b74b-f20aa83ee8fb None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:03.633042 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34913513-9655-40bb-b74b-f20aa83ee8fb None admin] PATH_INFO: `/v3/services` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:03.633914 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34913513-9655-40bb-b74b-f20aa83ee8fb None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:50:03.634419 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34913513-9655-40bb-b74b-f20aa83ee8fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:03.636140 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34913513-9655-40bb-b74b-f20aa83ee8fb None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:03.642532 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 343/694] 10.222.0.39 () {56 vars in 1126 bytes} [Fri Oct 14 02:50:03 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:50:03.643366 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34913513-9655-40bb-b74b-f20aa83ee8fb None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:03.679903 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:03.710545 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:03.733654 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a21957a-afd7-414f-98f1-f68b215febab None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:03.747864 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34913513-9655-40bb-b74b-f20aa83ee8fb None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:03.767500 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a72463-cbd8-46a7-ae61-7505306b6d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:03.768166 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a72463-cbd8-46a7-ae61-7505306b6d16 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:03.768450 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a72463-cbd8-46a7-ae61-7505306b6d16 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:03.769241 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a72463-cbd8-46a7-ae61-7505306b6d16 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:03.769966 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a72463-cbd8-46a7-ae61-7505306b6d16 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:03.771479 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a72463-cbd8-46a7-ae61-7505306b6d16 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:03.780641 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-31a72463-cbd8-46a7-ae61-7505306b6d16 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 14 02:50:03.780641 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:50:03.780641 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 14 02:50:03.780641 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 14 02:50:03.780641 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 14 02:50:03.780641 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 14 02:50:03.780641 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 14 02:50:03.780641 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:50:03.780641 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 14 02:50:03.780641 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:50:03.780641 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 02:50:03.780641 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:50:03.780641 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:50:03.780641 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:50:03.780641 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:50:03.780641 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:50:03.780641 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:50:03.781591 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:50:03.781591 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:50:03.781591 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:50:03.781591 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:50:03.781591 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:50:03.781591 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:50:03.781591 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 14 02:50:03.781591 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 14 02:50:03.781591 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 14 02:50:03.781591 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 14 02:50:03.781591 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:50:03.781591 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:50:03.781591 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:50:03.781591 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:50:03.781591 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:50:03.781591 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:50:03.783030 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:50:03.783030 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:50:03.783030 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:50:03.783030 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:50:03.783030 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:50:03.783030 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:50:03.783030 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:50:03.783030 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:50:03.783030 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:50:03.783030 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:50:03.783030 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 14 02:50:03.783030 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:50:03.783030 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 14 02:50:03.783030 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:50:03.783785 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d193e299-0ab9-417d-b2a6-9423b2d7bb3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:03.783785 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d193e299-0ab9-417d-b2a6-9423b2d7bb3d None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:03.783785 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d193e299-0ab9-417d-b2a6-9423b2d7bb3d None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:03.783991 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 352/695] 10.222.0.39 () {56 vars in 1114 bytes} [Fri Oct 14 02:50:03 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:50:03.783991 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a72463-cbd8-46a7-ae61-7505306b6d16 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:03.787080 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d193e299-0ab9-417d-b2a6-9423b2d7bb3d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:03.857622 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d193e299-0ab9-417d-b2a6-9423b2d7bb3d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:03.859290 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d193e299-0ab9-417d-b2a6-9423b2d7bb3d None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:03.882680 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a72463-cbd8-46a7-ae61-7505306b6d16 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:03.900120 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:03.920568 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b688b9a-82d0-4806-af4a-1e22681d0f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:03.921235 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b688b9a-82d0-4806-af4a-1e22681d0f13 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:03.921485 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b688b9a-82d0-4806-af4a-1e22681d0f13 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:03.921779 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b688b9a-82d0-4806-af4a-1e22681d0f13 None admin] PATH_INFO: `/v3/services` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:03.922442 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b688b9a-82d0-4806-af4a-1e22681d0f13 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:50:03.922753 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b688b9a-82d0-4806-af4a-1e22681d0f13 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:03.924129 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b688b9a-82d0-4806-af4a-1e22681d0f13 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:03.927740 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:03.934160 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 353/696] 10.222.0.39 () {56 vars in 1126 bytes} [Fri Oct 14 02:50:03 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:50:03.935028 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b688b9a-82d0-4806-af4a-1e22681d0f13 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:03.965102 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 344/697] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Oct 14 02:50:03 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 14 02:50:03.965893 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d193e299-0ab9-417d-b2a6-9423b2d7bb3d None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:04.050256 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b688b9a-82d0-4806-af4a-1e22681d0f13 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:04.079751 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d193e299-0ab9-417d-b2a6-9423b2d7bb3d None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:04.087641 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9920da-dece-4cb7-a360-f1c5c944ed1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:04.088241 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9920da-dece-4cb7-a360-f1c5c944ed1b None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:04.088521 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9920da-dece-4cb7-a360-f1c5c944ed1b None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:04.088792 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9920da-dece-4cb7-a360-f1c5c944ed1b None admin] PATH_INFO: `/v3/services` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:04.089414 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9920da-dece-4cb7-a360-f1c5c944ed1b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:50:04.089913 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9920da-dece-4cb7-a360-f1c5c944ed1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:04.091315 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9920da-dece-4cb7-a360-f1c5c944ed1b None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:04.100628 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 354/698] 10.222.0.39 () {56 vars in 1114 bytes} [Fri Oct 14 02:50:03 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:50:04.101459 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9920da-dece-4cb7-a360-f1c5c944ed1b None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:04.115893 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a0d42b-8cf9-47d3-a352-e5925ac4f096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:04.116728 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a0d42b-8cf9-47d3-a352-e5925ac4f096 None admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:04.117019 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a0d42b-8cf9-47d3-a352-e5925ac4f096 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:04.117660 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a0d42b-8cf9-47d3-a352-e5925ac4f096 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:04.118376 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a0d42b-8cf9-47d3-a352-e5925ac4f096 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:04.119763 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a0d42b-8cf9-47d3-a352-e5925ac4f096 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:04.143566 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:04.199417 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9920da-dece-4cb7-a360-f1c5c944ed1b None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:04.207036 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 345/699] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 02:50:03 2022] POST /identity/v3/endpoints => generated 347 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:50:04.207756 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a0d42b-8cf9-47d3-a352-e5925ac4f096 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:04.244391 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:04.259124 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548febdb-092e-48d5-a2f6-ed3073b3affb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:04.259767 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548febdb-092e-48d5-a2f6-ed3073b3affb None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:04.260092 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548febdb-092e-48d5-a2f6-ed3073b3affb None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:04.260371 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548febdb-092e-48d5-a2f6-ed3073b3affb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:04.273906 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548febdb-092e-48d5-a2f6-ed3073b3affb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:04.275548 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548febdb-092e-48d5-a2f6-ed3073b3affb None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:04.292746 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-548febdb-092e-48d5-a2f6-ed3073b3affb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:50:04.292746 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:50:04.292746 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:50:04.292746 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:50:04.292746 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:50:04.292746 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:50:04.292746 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:50:04.292746 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:50:04.292746 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:50:04.292746 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:50:04.292746 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:50:04.292746 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:50:04.292746 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:50:04.292746 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:50:04.292746 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:50:04.292746 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:50:04.293799 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:50:04.293799 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:50:04.293799 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:50:04.293799 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:50:04.293799 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:50:04.293799 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:50:04.293799 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:50:04.293799 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:50:04.293799 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:50:04.293799 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:50:04.293799 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:50:04.293799 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:50:04.293799 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:50:04.293799 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:50:04.293799 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:50:04.293799 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:50:04.293799 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:50:04.295132 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:50:04.295132 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:50:04.295132 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:50:04.295132 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 355/700] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:50:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:50:04.301261 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548febdb-092e-48d5-a2f6-ed3073b3affb None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:04.351476 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a0d42b-8cf9-47d3-a352-e5925ac4f096 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:04.394626 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab83cea-5bc8-4615-b94f-4e5ca424de79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:04.395530 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab83cea-5bc8-4615-b94f-4e5ca424de79 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:04.396000 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab83cea-5bc8-4615-b94f-4e5ca424de79 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:04.396612 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab83cea-5bc8-4615-b94f-4e5ca424de79 None admin] PATH_INFO: `/v3/services` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:04.397253 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab83cea-5bc8-4615-b94f-4e5ca424de79 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:50:04.398412 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab83cea-5bc8-4615-b94f-4e5ca424de79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:04.399795 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab83cea-5bc8-4615-b94f-4e5ca424de79 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:04.405501 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 346/701] 10.222.0.39 () {56 vars in 1114 bytes} [Fri Oct 14 02:50:04 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:50:04.411807 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab83cea-5bc8-4615-b94f-4e5ca424de79 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:04.423282 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:04.438073 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548febdb-092e-48d5-a2f6-ed3073b3affb None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:04.476932 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c815191-d723-4548-8f21-c8021459837c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:04.477611 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c815191-d723-4548-8f21-c8021459837c None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:04.477926 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c815191-d723-4548-8f21-c8021459837c None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:04.478246 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c815191-d723-4548-8f21-c8021459837c None admin] PATH_INFO: `/v3/domains` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:04.478889 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c815191-d723-4548-8f21-c8021459837c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:50:04.479443 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c815191-d723-4548-8f21-c8021459837c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:04.480813 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c815191-d723-4548-8f21-c8021459837c None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:04.492600 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 356/702] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Oct 14 02:50:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:50:04.498776 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c815191-d723-4548-8f21-c8021459837c None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:04.525945 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab83cea-5bc8-4615-b94f-4e5ca424de79 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:04.534100 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:04.563789 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1495529b-da97-4649-8e6e-871e0b6e4ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:04.564611 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1495529b-da97-4649-8e6e-871e0b6e4ea1 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:04.564733 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1495529b-da97-4649-8e6e-871e0b6e4ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:04.564940 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1495529b-da97-4649-8e6e-871e0b6e4ea1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:04.565726 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1495529b-da97-4649-8e6e-871e0b6e4ea1 None admin] RBAC: Adding query filter params (interface=public, service_id=ad5d8a34b2e049b6b2fbc8f9053a2a02, region_id=RegionOne) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:50:04.566002 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1495529b-da97-4649-8e6e-871e0b6e4ea1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:04.567532 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1495529b-da97-4649-8e6e-871e0b6e4ea1 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:04.573409 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 347/703] 10.222.0.39 () {56 vars in 1251 bytes} [Fri Oct 14 02:50:04 2022] GET /identity/v3/endpoints?service_id=ad5d8a34b2e049b6b2fbc8f9053a2a02&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:50:04.579502 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1495529b-da97-4649-8e6e-871e0b6e4ea1 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:04.615199 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c815191-d723-4548-8f21-c8021459837c None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:04.652977 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f4c20e-7a11-44e8-91d8-028434f908b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:04.653986 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f4c20e-7a11-44e8-91d8-028434f908b6 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:04.654422 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f4c20e-7a11-44e8-91d8-028434f908b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:04.654900 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f4c20e-7a11-44e8-91d8-028434f908b6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:04.662539 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f4c20e-7a11-44e8-91d8-028434f908b6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:04.664281 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f4c20e-7a11-44e8-91d8-028434f908b6 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:04.676822 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-33f4c20e-7a11-44e8-91d8-028434f908b6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:50:04.676822 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:50:04.676822 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:50:04.676822 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:50:04.676822 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:50:04.676822 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:50:04.676822 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:50:04.676822 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:50:04.676822 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:50:04.676822 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:50:04.676822 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:50:04.676822 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:50:04.676822 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 14 02:50:04.676822 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 14 02:50:04.676822 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 14 02:50:04.677926 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 14 02:50:04.677926 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:50:04.677926 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:50:04.677926 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:50:04.677926 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:50:04.677926 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:50:04.677926 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:50:04.677926 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:50:04.677926 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:50:04.677926 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:50:04.677926 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:50:04.677926 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:50:04.677926 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:50:04.677926 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:50:04.677926 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:50:04.677926 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:50:04.677926 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:50:04.679009 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 14 02:50:04.679009 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 14 02:50:04.679009 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 14 02:50:04.679009 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 14 02:50:04.679009 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 14 02:50:04.679009 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 14 02:50:04.679009 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:50:04.679009 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:50:04.685652 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 357/704] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Oct 14 02:50:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:50:04.686765 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f4c20e-7a11-44e8-91d8-028434f908b6 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:04.695372 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1495529b-da97-4649-8e6e-871e0b6e4ea1 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:04.699657 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:04.731593 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0443e9b-1257-4ff0-a4c9-18bd4178a021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:04.733513 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0443e9b-1257-4ff0-a4c9-18bd4178a021 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:04.733997 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0443e9b-1257-4ff0-a4c9-18bd4178a021 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:04.734436 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0443e9b-1257-4ff0-a4c9-18bd4178a021 None admin] PATH_INFO: `/v3/services` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:04.735268 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0443e9b-1257-4ff0-a4c9-18bd4178a021 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:04.736887 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0443e9b-1257-4ff0-a4c9-18bd4178a021 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:04.743989 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 348/705] 10.222.0.39 () {56 vars in 1087 bytes} [Fri Oct 14 02:50:04 2022] GET /identity/v3/services => generated 1497 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 02:50:04.809253 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f4c20e-7a11-44e8-91d8-028434f908b6 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:04.849199 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf66ba8-78a7-48dd-ba08-5704d8b2bd97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:04.849842 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf66ba8-78a7-48dd-ba08-5704d8b2bd97 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:04.850125 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf66ba8-78a7-48dd-ba08-5704d8b2bd97 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:04.850395 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf66ba8-78a7-48dd-ba08-5704d8b2bd97 None admin] PATH_INFO: `/v3/projects` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:04.851174 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf66ba8-78a7-48dd-ba08-5704d8b2bd97 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:50:04.851483 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf66ba8-78a7-48dd-ba08-5704d8b2bd97 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:04.853071 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf66ba8-78a7-48dd-ba08-5704d8b2bd97 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:04.868516 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 358/706] 10.222.0.39 () {56 vars in 1148 bytes} [Fri Oct 14 02:50:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:50:04.874740 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0443e9b-1257-4ff0-a4c9-18bd4178a021 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:04.892904 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:04.915248 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7657d29-83bd-4fb4-a203-10ace6da4c2a None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:04.915248 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7657d29-83bd-4fb4-a203-10ace6da4c2a None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:04.915248 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7657d29-83bd-4fb4-a203-10ace6da4c2a None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:04.915813 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 359/707] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:50:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 14 02:50:04.922171 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47b0403-3d59-4a93-8a4b-4ce59e815d26 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:04.922461 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47b0403-3d59-4a93-8a4b-4ce59e815d26 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:04.922906 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47b0403-3d59-4a93-8a4b-4ce59e815d26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:04.945965 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-a47b0403-3d59-4a93-8a4b-4ce59e815d26 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:50:04.947161 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-a47b0403-3d59-4a93-8a4b-4ce59e815d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:50:04.993243 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0443e9b-1257-4ff0-a4c9-18bd4178a021 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:05.019203 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:05.025031 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 360/708] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:50:04 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 14 02:50:05.043749 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7295d70a-0400-456c-8268-8472ec204455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:05.044355 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7295d70a-0400-456c-8268-8472ec204455 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:05.044635 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7295d70a-0400-456c-8268-8472ec204455 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:05.044912 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7295d70a-0400-456c-8268-8472ec204455 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:05.045631 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7295d70a-0400-456c-8268-8472ec204455 None admin] RBAC: Adding query filter params (role.id=22dff4ef49264f709f154e98d331d850, scope.project.id=dac9f8cd285c44e889a621b8b7fe479e, user.id=44b37050076b42e6af1a67bd7d6d06e1) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:50:05.045964 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7295d70a-0400-456c-8268-8472ec204455 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:05.047444 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7295d70a-0400-456c-8268-8472ec204455 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:05.054822 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 349/709] 10.222.0.39 () {56 vars in 1374 bytes} [Fri Oct 14 02:50:04 2022] GET /identity/v3/role_assignments?user.id=44b37050076b42e6af1a67bd7d6d06e1&scope.project.id=dac9f8cd285c44e889a621b8b7fe479e&role.id=22dff4ef49264f709f154e98d331d850 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:50:05.307200 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47b0403-3d59-4a93-8a4b-4ce59e815d26 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:05.408699 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47b0403-3d59-4a93-8a4b-4ce59e815d26 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:05.444601 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c526906a-0259-4f7a-a586-42b4fb5a17cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:05.445237 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c526906a-0259-4f7a-a586-42b4fb5a17cd None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:05.445514 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c526906a-0259-4f7a-a586-42b4fb5a17cd None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:05.445805 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c526906a-0259-4f7a-a586-42b4fb5a17cd None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:05.446390 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c526906a-0259-4f7a-a586-42b4fb5a17cd None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:05.447754 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c526906a-0259-4f7a-a586-42b4fb5a17cd None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:05.457201 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-c526906a-0259-4f7a-a586-42b4fb5a17cd None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 14 02:50:05.457201 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:50:05.457201 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 14 02:50:05.457201 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 14 02:50:05.457201 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 14 02:50:05.457201 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 14 02:50:05.457201 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 14 02:50:05.457201 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:50:05.457201 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 14 02:50:05.457201 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:50:05.457201 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 02:50:05.457201 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:50:05.457201 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:50:05.457201 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:50:05.457201 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:50:05.457201 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:50:05.457201 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:50:05.459482 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:50:05.459482 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:50:05.459482 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:50:05.459482 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:50:05.459482 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:50:05.459482 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:50:05.459482 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 14 02:50:05.459482 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 14 02:50:05.459482 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 14 02:50:05.459482 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 14 02:50:05.459482 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:50:05.459482 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:50:05.459482 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:50:05.459482 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:50:05.459482 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:50:05.459482 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:50:05.461450 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:50:05.461450 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:50:05.461450 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:50:05.461450 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:50:05.461450 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:50:05.461450 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:50:05.461450 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:50:05.461450 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:50:05.461450 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:50:05.461450 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:50:05.461450 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 14 02:50:05.461450 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:50:05.461450 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 14 02:50:05.461450 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:50:05.469799 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 361/710] 10.222.0.39 () {56 vars in 1126 bytes} [Fri Oct 14 02:50:05 2022] GET /identity/v3/services/object-store => generated 93 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:50:05.470046 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7295d70a-0400-456c-8268-8472ec204455 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:05.545898 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:05.570549 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7295d70a-0400-456c-8268-8472ec204455 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:05.602948 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e20844-6409-418a-8f56-b7ef456b22e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:05.603892 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e20844-6409-418a-8f56-b7ef456b22e4 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:05.604648 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e20844-6409-418a-8f56-b7ef456b22e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:05.605137 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e20844-6409-418a-8f56-b7ef456b22e4 None admin] PATH_INFO: `/v3/services` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:05.606045 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e20844-6409-418a-8f56-b7ef456b22e4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:50:05.606524 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e20844-6409-418a-8f56-b7ef456b22e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:05.608084 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e20844-6409-418a-8f56-b7ef456b22e4 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:05.617944 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 350/711] 10.222.0.39 () {56 vars in 1122 bytes} [Fri Oct 14 02:50:05 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:50:05.619476 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c526906a-0259-4f7a-a586-42b4fb5a17cd None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:05.683413 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:05.724927 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c526906a-0259-4f7a-a586-42b4fb5a17cd None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:05.762082 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6a7fd4-7aa2-41cb-8ab9-d02563c1e000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:05.762691 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6a7fd4-7aa2-41cb-8ab9-d02563c1e000 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:05.762972 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6a7fd4-7aa2-41cb-8ab9-d02563c1e000 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:05.763243 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6a7fd4-7aa2-41cb-8ab9-d02563c1e000 None admin] PATH_INFO: `/v3/services` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:05.763877 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6a7fd4-7aa2-41cb-8ab9-d02563c1e000 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:50:05.764190 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6a7fd4-7aa2-41cb-8ab9-d02563c1e000 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:05.765562 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6a7fd4-7aa2-41cb-8ab9-d02563c1e000 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:05.770944 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 362/712] 10.222.0.39 () {56 vars in 1122 bytes} [Fri Oct 14 02:50:05 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:50:05.776525 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e20844-6409-418a-8f56-b7ef456b22e4 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:05.887966 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e20844-6409-418a-8f56-b7ef456b22e4 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:05.921820 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eee109-07fb-4c73-b7ae-1e01868b27ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:05.922434 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eee109-07fb-4c73-b7ae-1e01868b27ac None admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:05.922712 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eee109-07fb-4c73-b7ae-1e01868b27ac None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:05.922984 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eee109-07fb-4c73-b7ae-1e01868b27ac None admin] PATH_INFO: `/v3/endpoints` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:05.923659 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61eee109-07fb-4c73-b7ae-1e01868b27ac None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:05.925525 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61eee109-07fb-4c73-b7ae-1e01868b27ac None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:06.001709 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:06.012637 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 351/713] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 02:50:05 2022] POST /identity/v3/endpoints => generated 349 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:50:06.479935 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca4b2f-4f09-4049-8d5f-bcb582485d18 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:06.480457 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca4b2f-4f09-4049-8d5f-bcb582485d18 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:06.480457 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca4b2f-4f09-4049-8d5f-bcb582485d18 None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:06.482084 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 363/714] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:50:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:50:06.488068 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cf54cd-c652-4c8d-8010-cdc4bbcda4c1 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:06.488477 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cf54cd-c652-4c8d-8010-cdc4bbcda4c1 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:06.488648 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cf54cd-c652-4c8d-8010-cdc4bbcda4c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:06.508666 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-03cf54cd-c652-4c8d-8010-cdc4bbcda4c1 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:50:06.509882 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-03cf54cd-c652-4c8d-8010-cdc4bbcda4c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:50:06.585059 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 352/715] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:50:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:50:06.852268 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaca4b2f-4f09-4049-8d5f-bcb582485d18 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:06.961066 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaca4b2f-4f09-4049-8d5f-bcb582485d18 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:07.000394 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:07.004364 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76aff36-2a13-4f06-9a7c-78e576f72da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:07.005263 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76aff36-2a13-4f06-9a7c-78e576f72da8 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:07.005784 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76aff36-2a13-4f06-9a7c-78e576f72da8 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:07.006268 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76aff36-2a13-4f06-9a7c-78e576f72da8 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:07.007160 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76aff36-2a13-4f06-9a7c-78e576f72da8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:07.008743 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76aff36-2a13-4f06-9a7c-78e576f72da8 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:07.018031 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-b76aff36-2a13-4f06-9a7c-78e576f72da8 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 14 02:50:07.018031 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:50:07.018031 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 14 02:50:07.018031 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 14 02:50:07.018031 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 14 02:50:07.018031 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 14 02:50:07.018031 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 14 02:50:07.018031 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:50:07.018031 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 14 02:50:07.018031 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:50:07.018031 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 02:50:07.018031 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:50:07.018031 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:50:07.018031 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:50:07.018031 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:50:07.018031 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:50:07.018031 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:50:07.030124 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:50:07.030124 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:50:07.030124 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:50:07.030124 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:50:07.030124 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:50:07.030124 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:50:07.030124 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 14 02:50:07.030124 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 14 02:50:07.030124 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 14 02:50:07.030124 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 14 02:50:07.030124 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:50:07.030124 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:50:07.030124 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:50:07.030124 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:50:07.030124 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:50:07.030124 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:50:07.030985 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:50:07.030985 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:50:07.030985 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:50:07.030985 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:50:07.030985 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:50:07.030985 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:50:07.030985 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:50:07.030985 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:50:07.030985 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:50:07.030985 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:50:07.030985 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 14 02:50:07.030985 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:50:07.030985 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 14 02:50:07.030985 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:50:07.030985 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 364/716] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Oct 14 02:50:06 2022] GET /identity/v3/services/compute => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:50:07.031770 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cf54cd-c652-4c8d-8010-cdc4bbcda4c1 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:07.077642 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77790529-9384-43e5-8683-692bd588f20a None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:07.077983 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77790529-9384-43e5-8683-692bd588f20a None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:07.078320 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77790529-9384-43e5-8683-692bd588f20a None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:07.079976 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 365/717] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:50:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:50:07.084985 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27eeded-e368-4fcc-b16e-7b60bfd7b823 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:07.085271 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27eeded-e368-4fcc-b16e-7b60bfd7b823 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:07.085546 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27eeded-e368-4fcc-b16e-7b60bfd7b823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:07.105493 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-d27eeded-e368-4fcc-b16e-7b60bfd7b823 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:50:07.106832 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-d27eeded-e368-4fcc-b16e-7b60bfd7b823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:50:07.138125 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cf54cd-c652-4c8d-8010-cdc4bbcda4c1 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:07.171286 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:07.183172 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 366/718] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:50:07 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:50:07.185484 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af11d053-4aec-4067-8f4c-aaf0b576b974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:07.193805 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af11d053-4aec-4067-8f4c-aaf0b576b974 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:07.193805 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af11d053-4aec-4067-8f4c-aaf0b576b974 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:07.193805 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af11d053-4aec-4067-8f4c-aaf0b576b974 None admin] PATH_INFO: `/v3/services` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:07.193805 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af11d053-4aec-4067-8f4c-aaf0b576b974 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:50:07.193805 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af11d053-4aec-4067-8f4c-aaf0b576b974 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:07.193805 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af11d053-4aec-4067-8f4c-aaf0b576b974 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:07.195162 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 353/719] 10.222.0.39 () {56 vars in 1112 bytes} [Fri Oct 14 02:50:07 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:50:07.199566 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27eeded-e368-4fcc-b16e-7b60bfd7b823 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:07.314483 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27eeded-e368-4fcc-b16e-7b60bfd7b823 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:07.335977 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:07.349036 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d989bc11-94fc-4061-9e9e-96876f3a2c0a None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:07.349343 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d989bc11-94fc-4061-9e9e-96876f3a2c0a None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:07.349725 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d989bc11-94fc-4061-9e9e-96876f3a2c0a None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:07.351266 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 354/720] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:50:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:50:07.351593 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729c8149-df23-43bd-91a9-3c89194dde41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:07.352207 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729c8149-df23-43bd-91a9-3c89194dde41 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:07.352549 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729c8149-df23-43bd-91a9-3c89194dde41 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:07.352823 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729c8149-df23-43bd-91a9-3c89194dde41 None admin] PATH_INFO: `/v3/services` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:07.353524 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729c8149-df23-43bd-91a9-3c89194dde41 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:50:07.353879 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729c8149-df23-43bd-91a9-3c89194dde41 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:07.355648 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729c8149-df23-43bd-91a9-3c89194dde41 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:07.358637 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d328c6f-bdaa-41ed-9e81-1342aaad580b None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:07.359146 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d328c6f-bdaa-41ed-9e81-1342aaad580b None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:07.359627 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d328c6f-bdaa-41ed-9e81-1342aaad580b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:07.361261 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 367/721] 10.222.0.39 () {56 vars in 1112 bytes} [Fri Oct 14 02:50:07 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:50:07.379912 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-5d328c6f-bdaa-41ed-9e81-1342aaad580b None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:50:07.381623 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-5d328c6f-bdaa-41ed-9e81-1342aaad580b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:50:07.447162 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729c8149-df23-43bd-91a9-3c89194dde41 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:07.459313 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 355/722] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:50:07 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 14 02:50:07.564236 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729c8149-df23-43bd-91a9-3c89194dde41 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:07.603444 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c52f548-5a9b-43f0-af22-7984c3687737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:07.604670 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c52f548-5a9b-43f0-af22-7984c3687737 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:07.605185 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c52f548-5a9b-43f0-af22-7984c3687737 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:07.605717 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c52f548-5a9b-43f0-af22-7984c3687737 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:07.606654 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c52f548-5a9b-43f0-af22-7984c3687737 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:07.608323 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c52f548-5a9b-43f0-af22-7984c3687737 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:07.617965 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-3c52f548-5a9b-43f0-af22-7984c3687737 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 14 02:50:07.617965 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:50:07.617965 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 14 02:50:07.617965 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 14 02:50:07.617965 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 14 02:50:07.617965 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 14 02:50:07.617965 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 14 02:50:07.617965 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:50:07.617965 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 14 02:50:07.617965 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:50:07.617965 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 02:50:07.617965 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:50:07.617965 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:50:07.617965 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:50:07.617965 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:50:07.617965 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:50:07.617965 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:50:07.618958 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:50:07.618958 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:50:07.618958 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:50:07.618958 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:50:07.618958 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:50:07.618958 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:50:07.618958 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 14 02:50:07.618958 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 14 02:50:07.618958 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 14 02:50:07.618958 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 14 02:50:07.618958 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:50:07.618958 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:50:07.618958 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:50:07.618958 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:50:07.618958 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:50:07.618958 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:50:07.619787 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:50:07.619787 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:50:07.619787 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:50:07.619787 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:50:07.619787 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:50:07.619787 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:50:07.619787 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:50:07.619787 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:50:07.619787 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:50:07.619787 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:50:07.619787 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 14 02:50:07.619787 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:50:07.619787 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 14 02:50:07.619787 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:50:07.622392 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 368/723] 10.222.0.39 () {56 vars in 1114 bytes} [Fri Oct 14 02:50:07 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 02:50:07.629839 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d328c6f-bdaa-41ed-9e81-1342aaad580b None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:07.736779 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d328c6f-bdaa-41ed-9e81-1342aaad580b None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:07.749463 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c52f548-5a9b-43f0-af22-7984c3687737 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:07.757528 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:07.780539 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f37527d-f868-4442-8c77-bf5d4df2c79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:07.788444 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f37527d-f868-4442-8c77-bf5d4df2c79c None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:07.788444 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f37527d-f868-4442-8c77-bf5d4df2c79c None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:07.788444 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f37527d-f868-4442-8c77-bf5d4df2c79c None admin] PATH_INFO: `/v3/services` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:07.788444 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f37527d-f868-4442-8c77-bf5d4df2c79c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:50:07.788444 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f37527d-f868-4442-8c77-bf5d4df2c79c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:07.788444 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f37527d-f868-4442-8c77-bf5d4df2c79c None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:07.792487 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 356/724] 10.222.0.39 () {56 vars in 1114 bytes} [Fri Oct 14 02:50:07 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:50:07.798095 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f37527d-f868-4442-8c77-bf5d4df2c79c None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:07.808479 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:07.877466 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c52f548-5a9b-43f0-af22-7984c3687737 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:07.916119 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f37527d-f868-4442-8c77-bf5d4df2c79c None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:07.916318 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9616b9fe-0c20-43cd-962d-1803b1f590bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:07.916751 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9616b9fe-0c20-43cd-962d-1803b1f590bb None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:07.917032 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9616b9fe-0c20-43cd-962d-1803b1f590bb None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:07.917300 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9616b9fe-0c20-43cd-962d-1803b1f590bb None admin] PATH_INFO: `/v3/services/image` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:07.917906 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9616b9fe-0c20-43cd-962d-1803b1f590bb None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:07.919398 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9616b9fe-0c20-43cd-962d-1803b1f590bb None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:07.929100 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-9616b9fe-0c20-43cd-962d-1803b1f590bb None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 14 02:50:07.929100 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:50:07.929100 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 14 02:50:07.929100 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 14 02:50:07.929100 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 14 02:50:07.929100 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 14 02:50:07.929100 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 14 02:50:07.929100 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:50:07.929100 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 14 02:50:07.929100 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:50:07.929100 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 02:50:07.929100 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:50:07.929100 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:50:07.929100 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:50:07.929100 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:50:07.929100 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:50:07.929100 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:50:07.930161 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:50:07.930161 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:50:07.930161 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:50:07.930161 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:50:07.930161 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:50:07.930161 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:50:07.930161 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 14 02:50:07.930161 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 14 02:50:07.930161 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 14 02:50:07.930161 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 14 02:50:07.930161 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:50:07.930161 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:50:07.930161 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:50:07.930161 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:50:07.930161 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:50:07.930161 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:50:07.931018 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:50:07.931018 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:50:07.931018 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:50:07.931018 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:50:07.931018 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:50:07.931018 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:50:07.931018 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:50:07.931018 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:50:07.931018 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:50:07.931018 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:50:07.931018 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 14 02:50:07.931018 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:50:07.931018 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 14 02:50:07.931018 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:50:07.935334 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 369/725] 10.222.0.39 () {56 vars in 1105 bytes} [Fri Oct 14 02:50:07 2022] GET /identity/v3/services/image => generated 86 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:50:07.936757 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9616b9fe-0c20-43cd-962d-1803b1f590bb None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:07.963588 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e81a12-2587-4c34-8a05-8e6a8eac72d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:07.964518 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e81a12-2587-4c34-8a05-8e6a8eac72d0 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:07.965235 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e81a12-2587-4c34-8a05-8e6a8eac72d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:07.965719 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e81a12-2587-4c34-8a05-8e6a8eac72d0 None admin] PATH_INFO: `/v3/services` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:07.967176 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e81a12-2587-4c34-8a05-8e6a8eac72d0 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:50:07.967672 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e81a12-2587-4c34-8a05-8e6a8eac72d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:07.970121 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e81a12-2587-4c34-8a05-8e6a8eac72d0 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:07.977751 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 357/726] 10.222.0.39 () {56 vars in 1114 bytes} [Fri Oct 14 02:50:07 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:50:07.983822 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e81a12-2587-4c34-8a05-8e6a8eac72d0 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:08.040817 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:08.052557 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9616b9fe-0c20-43cd-962d-1803b1f590bb None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:08.067389 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:08.089945 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17621b9c-b24f-4706-b979-50d07b6c8bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:08.091123 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17621b9c-b24f-4706-b979-50d07b6c8bad None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:08.091701 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17621b9c-b24f-4706-b979-50d07b6c8bad None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:08.092176 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17621b9c-b24f-4706-b979-50d07b6c8bad None admin] PATH_INFO: `/v3/services` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:08.092978 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17621b9c-b24f-4706-b979-50d07b6c8bad None admin] RBAC: Adding query filter params (name=image) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:50:08.093505 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17621b9c-b24f-4706-b979-50d07b6c8bad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:08.095082 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17621b9c-b24f-4706-b979-50d07b6c8bad None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:08.098641 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e81a12-2587-4c34-8a05-8e6a8eac72d0 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:08.100911 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 370/727] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:50:07 2022] GET /identity/v3/services?name=image => generated 122 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:50:08.107816 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17621b9c-b24f-4706-b979-50d07b6c8bad None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:08.149388 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115b2cd8-87a4-4ae9-82de-f5707e20f432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:08.150564 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115b2cd8-87a4-4ae9-82de-f5707e20f432 None admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:08.150846 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115b2cd8-87a4-4ae9-82de-f5707e20f432 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:08.151925 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115b2cd8-87a4-4ae9-82de-f5707e20f432 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:08.152643 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115b2cd8-87a4-4ae9-82de-f5707e20f432 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:08.154564 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115b2cd8-87a4-4ae9-82de-f5707e20f432 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:08.232692 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17621b9c-b24f-4706-b979-50d07b6c8bad None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:08.238558 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 358/728] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 02:50:07 2022] POST /identity/v3/endpoints => generated 346 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 02:50:08.269811 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d88ed1-4e33-4bd2-b271-4668b9e1a4e5 None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:08.270147 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d88ed1-4e33-4bd2-b271-4668b9e1a4e5 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:08.270380 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d88ed1-4e33-4bd2-b271-4668b9e1a4e5 None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:08.272079 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 359/729] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:50:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 14 02:50:08.278054 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f0bae5-8b6c-46c0-a916-bf99d9142ebe None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:08.278399 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f0bae5-8b6c-46c0-a916-bf99d9142ebe None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:08.278600 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f0bae5-8b6c-46c0-a916-bf99d9142ebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:08.288131 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59a1fc6-abb8-4949-ac40-b5fbe4406ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:08.289260 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59a1fc6-abb8-4949-ac40-b5fbe4406ff9 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:08.289794 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59a1fc6-abb8-4949-ac40-b5fbe4406ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:08.290305 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59a1fc6-abb8-4949-ac40-b5fbe4406ff9 None admin] PATH_INFO: `/v3/services` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:08.291235 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59a1fc6-abb8-4949-ac40-b5fbe4406ff9 None admin] RBAC: Adding query filter params (type=image) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:50:08.291762 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59a1fc6-abb8-4949-ac40-b5fbe4406ff9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:08.293831 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59a1fc6-abb8-4949-ac40-b5fbe4406ff9 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:08.300239 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 371/730] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:50:08 2022] GET /identity/v3/services?type=image => generated 122 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:50:08.301781 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-c7f0bae5-8b6c-46c0-a916-bf99d9142ebe None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:50:08.303020 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-c7f0bae5-8b6c-46c0-a916-bf99d9142ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:50:08.384479 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 360/731] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:50:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:50:08.657848 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59a1fc6-abb8-4949-ac40-b5fbe4406ff9 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:08.696744 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:08.765209 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59a1fc6-abb8-4949-ac40-b5fbe4406ff9 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:08.802762 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02ee9d6-362a-4353-bc5c-d96f1af87734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:08.804228 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02ee9d6-362a-4353-bc5c-d96f1af87734 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:08.804752 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02ee9d6-362a-4353-bc5c-d96f1af87734 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:08.805241 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02ee9d6-362a-4353-bc5c-d96f1af87734 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:08.806069 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02ee9d6-362a-4353-bc5c-d96f1af87734 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:08.807668 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02ee9d6-362a-4353-bc5c-d96f1af87734 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:08.816921 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-f02ee9d6-362a-4353-bc5c-d96f1af87734 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 14 02:50:08.816921 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:50:08.816921 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 14 02:50:08.816921 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 14 02:50:08.816921 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 14 02:50:08.816921 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 14 02:50:08.816921 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 14 02:50:08.816921 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:50:08.816921 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 14 02:50:08.816921 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:50:08.816921 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 02:50:08.816921 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:50:08.816921 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:50:08.816921 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:50:08.816921 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:50:08.816921 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:50:08.816921 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:50:08.817976 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:50:08.817976 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:50:08.817976 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:50:08.817976 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:50:08.817976 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:50:08.817976 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:50:08.817976 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 14 02:50:08.817976 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 14 02:50:08.817976 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 14 02:50:08.817976 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 14 02:50:08.817976 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:50:08.817976 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:50:08.817976 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:50:08.817976 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:50:08.817976 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:50:08.817976 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:50:08.818826 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:50:08.818826 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:50:08.818826 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:50:08.818826 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:50:08.818826 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:50:08.818826 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:50:08.818826 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:50:08.818826 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:50:08.818826 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:50:08.818826 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:50:08.818826 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 14 02:50:08.818826 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:50:08.818826 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 14 02:50:08.818826 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:50:08.821529 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 372/732] 10.222.0.39 () {56 vars in 1126 bytes} [Fri Oct 14 02:50:08 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:50:08.827177 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f0bae5-8b6c-46c0-a916-bf99d9142ebe None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:08.859693 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:08.935471 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f0bae5-8b6c-46c0-a916-bf99d9142ebe None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:08.970063 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b528a2-cf3d-4850-98a6-460aef94dce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:08.970738 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b528a2-cf3d-4850-98a6-460aef94dce5 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:08.971024 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b528a2-cf3d-4850-98a6-460aef94dce5 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:08.971301 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b528a2-cf3d-4850-98a6-460aef94dce5 None admin] PATH_INFO: `/v3/services` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:08.972586 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b528a2-cf3d-4850-98a6-460aef94dce5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:50:08.972897 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b528a2-cf3d-4850-98a6-460aef94dce5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:08.974312 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b528a2-cf3d-4850-98a6-460aef94dce5 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:08.979887 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 361/733] 10.222.0.39 () {56 vars in 1122 bytes} [Fri Oct 14 02:50:08 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:50:08.987368 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02ee9d6-362a-4353-bc5c-d96f1af87734 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:09.101210 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:09.104613 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02ee9d6-362a-4353-bc5c-d96f1af87734 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:09.140546 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4909654-d391-45f7-b222-d06744488901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:09.141208 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4909654-d391-45f7-b222-d06744488901 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:09.141494 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4909654-d391-45f7-b222-d06744488901 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:09.141786 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4909654-d391-45f7-b222-d06744488901 None admin] PATH_INFO: `/v3/services` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:09.142500 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4909654-d391-45f7-b222-d06744488901 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:50:09.142812 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4909654-d391-45f7-b222-d06744488901 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:09.144824 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4909654-d391-45f7-b222-d06744488901 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:09.150284 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 373/734] 10.222.0.39 () {56 vars in 1122 bytes} [Fri Oct 14 02:50:08 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:50:09.156355 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b528a2-cf3d-4850-98a6-460aef94dce5 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:09.261120 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:09.267399 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b528a2-cf3d-4850-98a6-460aef94dce5 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:09.303243 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a91ff3-ca1a-416b-93ed-5dec39690e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:09.303858 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a91ff3-ca1a-416b-93ed-5dec39690e89 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:09.304136 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a91ff3-ca1a-416b-93ed-5dec39690e89 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:09.304492 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a91ff3-ca1a-416b-93ed-5dec39690e89 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:09.305195 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a91ff3-ca1a-416b-93ed-5dec39690e89 None admin] RBAC: Adding query filter params (interface=admin, service_id=fc656af825124aca9eed0a471c2f9581, region_id=RegionOne) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:50:09.305507 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a91ff3-ca1a-416b-93ed-5dec39690e89 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:09.306905 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a91ff3-ca1a-416b-93ed-5dec39690e89 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:09.312543 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 362/735] 10.222.0.39 () {56 vars in 1249 bytes} [Fri Oct 14 02:50:09 2022] GET /identity/v3/endpoints?service_id=fc656af825124aca9eed0a471c2f9581&interface=admin®ion_id=RegionOne => generated 193 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:50:09.318658 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4909654-d391-45f7-b222-d06744488901 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:09.420993 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4909654-d391-45f7-b222-d06744488901 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:09.457519 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e7ff36-b634-4397-ab82-845096dc5c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:09.458188 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e7ff36-b634-4397-ab82-845096dc5c40 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:09.458470 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e7ff36-b634-4397-ab82-845096dc5c40 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:09.458746 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e7ff36-b634-4397-ab82-845096dc5c40 None admin] PATH_INFO: `/v3/services` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:09.459372 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e7ff36-b634-4397-ab82-845096dc5c40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:09.460804 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e7ff36-b634-4397-ab82-845096dc5c40 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:09.468403 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 374/736] 10.222.0.39 () {56 vars in 1087 bytes} [Fri Oct 14 02:50:09 2022] GET /identity/v3/services => generated 1497 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:50:09.587737 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d71cd-bb0f-457c-8225-35cb1b804d24 None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:09.588025 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d71cd-bb0f-457c-8225-35cb1b804d24 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:09.588310 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d71cd-bb0f-457c-8225-35cb1b804d24 None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:09.589983 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 363/737] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:50:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:50:09.595868 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf4529f-d305-4f6b-a8ed-e0eed856dc5e None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:09.596155 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf4529f-d305-4f6b-a8ed-e0eed856dc5e None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:09.596426 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf4529f-d305-4f6b-a8ed-e0eed856dc5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:09.617100 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-1bf4529f-d305-4f6b-a8ed-e0eed856dc5e None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:50:09.618382 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-1bf4529f-d305-4f6b-a8ed-e0eed856dc5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:50:09.644744 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:09.694358 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 375/738] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:50:09 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:50:09.963199 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1d71cd-bb0f-457c-8225-35cb1b804d24 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:10.074844 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1d71cd-bb0f-457c-8225-35cb1b804d24 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:10.110216 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67046ed4-9836-41c2-a53d-265a30632312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:10.110816 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67046ed4-9836-41c2-a53d-265a30632312 None admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:10.111101 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67046ed4-9836-41c2-a53d-265a30632312 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:10.111371 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67046ed4-9836-41c2-a53d-265a30632312 None admin] PATH_INFO: `/v3/services` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:10.112029 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67046ed4-9836-41c2-a53d-265a30632312 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:10.114092 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67046ed4-9836-41c2-a53d-265a30632312 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:10.219427 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 364/739] 10.222.0.39 () {60 vars in 1141 bytes} [Fri Oct 14 02:50:09 2022] POST /identity/v3/services => generated 242 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:50:10.572602 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0ac6f0-49d9-43aa-80ae-2a7c3567d295 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:10.572960 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0ac6f0-49d9-43aa-80ae-2a7c3567d295 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:10.573166 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0ac6f0-49d9-43aa-80ae-2a7c3567d295 None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:10.574801 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 376/740] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:50:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:50:10.580288 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bd2a28-4e10-48e1-bae7-016853c4ec05 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:10.580582 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bd2a28-4e10-48e1-bae7-016853c4ec05 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:10.581402 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bd2a28-4e10-48e1-bae7-016853c4ec05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:10.601205 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-74bd2a28-4e10-48e1-bae7-016853c4ec05 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:50:10.602452 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-74bd2a28-4e10-48e1-bae7-016853c4ec05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:50:10.617979 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:10.620956 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9852fbd3-800a-47a5-a768-56037da46701 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:10.621642 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9852fbd3-800a-47a5-a768-56037da46701 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:10.622454 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9852fbd3-800a-47a5-a768-56037da46701 None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:10.624302 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 377/741] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:50:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:50:10.629934 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cf1a80-17f7-4cf5-9ff4-acf3870bff24 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:10.630223 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cf1a80-17f7-4cf5-9ff4-acf3870bff24 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:10.632344 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cf1a80-17f7-4cf5-9ff4-acf3870bff24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:10.649311 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-a5cf1a80-17f7-4cf5-9ff4-acf3870bff24 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:50:10.650533 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-a5cf1a80-17f7-4cf5-9ff4-acf3870bff24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:50:10.682195 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 365/742] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:50:10 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:50:10.728796 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 378/743] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:50:10 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:50:10.968608 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bd2a28-4e10-48e1-bae7-016853c4ec05 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:10.995999 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cf1a80-17f7-4cf5-9ff4-acf3870bff24 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:11.090000 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:11.094235 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bd2a28-4e10-48e1-bae7-016853c4ec05 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:11.102216 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cf1a80-17f7-4cf5-9ff4-acf3870bff24 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:11.135524 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca3900f-92b0-4895-982f-97f50a1c38be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:11.136445 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca3900f-92b0-4895-982f-97f50a1c38be None admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:11.136827 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca3900f-92b0-4895-982f-97f50a1c38be None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:11.137197 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca3900f-92b0-4895-982f-97f50a1c38be None admin] PATH_INFO: `/v3/projects` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:11.138006 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca3900f-92b0-4895-982f-97f50a1c38be None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:11.139920 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca3900f-92b0-4895-982f-97f50a1c38be None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:11.140434 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8657983c-4b17-4366-8b6f-a265380b1fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:11.141214 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8657983c-4b17-4366-8b6f-a265380b1fb7 None admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:11.141582 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8657983c-4b17-4366-8b6f-a265380b1fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:11.141969 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8657983c-4b17-4366-8b6f-a265380b1fb7 None admin] PATH_INFO: `/v3/services` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:11.142648 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: WARNING keystone.server.flask.common [None req-0ca3900f-92b0-4895-982f-97f50a1c38be None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 14 02:50:11.142763 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8657983c-4b17-4366-8b6f-a265380b1fb7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:11.144335 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8657983c-4b17-4366-8b6f-a265380b1fb7 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:11.211834 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.sql.core [None req-0ca3900f-92b0-4895-982f-97f50a1c38be None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 14 02:50:11.211834 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 14 02:50:11.211834 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [parameters: {'id': '7a75332bed8146699dc185c50346057a', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 14 02:50:11.211834 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81518) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 14 02:50:11.221798 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-0ca3900f-92b0-4895-982f-97f50a1c38be None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7a75332bed8146699dc185c50346057a.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7a75332bed8146699dc185c50346057a. Oct 14 02:50:11.221798 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:50:11.221798 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 14 02:50:11.221798 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 14 02:50:11.221798 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 14 02:50:11.221798 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 14 02:50:11.221798 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 14 02:50:11.221798 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application result = self._query(query) Oct 14 02:50:11.221798 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 14 02:50:11.221798 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application conn.query(q) Oct 14 02:50:11.221798 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 14 02:50:11.221798 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 14 02:50:11.221798 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 14 02:50:11.222614 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application result.read() Oct 14 02:50:11.222614 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 14 02:50:11.222614 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 14 02:50:11.222614 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 14 02:50:11.222614 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 14 02:50:11.222614 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 14 02:50:11.222614 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 14 02:50:11.222614 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 14 02:50:11.222614 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 14 02:50:11.222614 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 14 02:50:11.222614 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:50:11.222614 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 14 02:50:11.222614 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:50:11.222614 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:50:11.222614 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 14 02:50:11.222614 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 14 02:50:11.222614 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 14 02:50:11.222614 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 14 02:50:11.223569 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 14 02:50:11.223569 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application next(self.gen) Oct 14 02:50:11.223569 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 14 02:50:11.223569 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application yield resource Oct 14 02:50:11.223569 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 14 02:50:11.223569 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application next(self.gen) Oct 14 02:50:11.223569 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 14 02:50:11.223569 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application self.session.rollback() Oct 14 02:50:11.223569 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 14 02:50:11.223569 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application self.force_reraise() Oct 14 02:50:11.223569 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 14 02:50:11.223569 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise self.value Oct 14 02:50:11.223569 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 14 02:50:11.223569 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 14 02:50:11.223569 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 14 02:50:11.223569 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application session.commit() Oct 14 02:50:11.223569 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 14 02:50:11.224472 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 14 02:50:11.224472 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 14 02:50:11.224472 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application self._prepare_impl() Oct 14 02:50:11.224472 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 14 02:50:11.224472 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application self.session.flush() Oct 14 02:50:11.224472 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 14 02:50:11.224472 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application self._flush(objects) Oct 14 02:50:11.224472 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 14 02:50:11.224472 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 14 02:50:11.224472 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 14 02:50:11.224472 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application compat.raise_( Oct 14 02:50:11.224472 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 14 02:50:11.224472 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception Oct 14 02:50:11.224472 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 14 02:50:11.224472 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application flush_context.execute() Oct 14 02:50:11.224472 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 14 02:50:11.224472 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rec.execute(self) Oct 14 02:50:11.226466 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 14 02:50:11.226466 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 14 02:50:11.226466 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 14 02:50:11.226466 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 14 02:50:11.226466 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 14 02:50:11.226466 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 14 02:50:11.226466 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 14 02:50:11.226466 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 14 02:50:11.226466 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 14 02:50:11.226466 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 14 02:50:11.226466 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 14 02:50:11.226466 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 14 02:50:11.226466 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 14 02:50:11.226466 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 14 02:50:11.226466 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 14 02:50:11.228357 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 14 02:50:11.228357 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 14 02:50:11.228357 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception Oct 14 02:50:11.228357 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 14 02:50:11.228357 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 14 02:50:11.228357 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 14 02:50:11.228357 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 14 02:50:11.228357 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 14 02:50:11.228357 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application result = self._query(query) Oct 14 02:50:11.228357 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 14 02:50:11.228357 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application conn.query(q) Oct 14 02:50:11.228357 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 14 02:50:11.228357 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 14 02:50:11.228357 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 14 02:50:11.228357 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application result.read() Oct 14 02:50:11.228357 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 14 02:50:11.228357 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 14 02:50:11.232014 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 14 02:50:11.232014 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 14 02:50:11.232014 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 14 02:50:11.232014 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 14 02:50:11.232014 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 14 02:50:11.232014 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 14 02:50:11.232014 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 14 02:50:11.232014 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 14 02:50:11.232014 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [parameters: {'id': '7a75332bed8146699dc185c50346057a', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 14 02:50:11.232014 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 14 02:50:11.232014 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:50:11.232014 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 02:50:11.232014 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:50:11.232014 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:50:11.232014 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 14 02:50:11.233899 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 14 02:50:11.233899 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 14 02:50:11.233899 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 14 02:50:11.233899 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Oct 14 02:50:11.233899 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:50:11.233899 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 02:50:11.233899 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:50:11.233899 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:50:11.233899 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:50:11.233899 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:50:11.233899 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:50:11.233899 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:50:11.233899 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:50:11.233899 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:50:11.233899 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:50:11.233899 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:50:11.233899 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:50:11.236581 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:50:11.236581 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Oct 14 02:50:11.236581 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Oct 14 02:50:11.236581 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:50:11.236581 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:50:11.236581 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 14 02:50:11.236581 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 14 02:50:11.236581 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7a75332bed8146699dc185c50346057a. Oct 14 02:50:11.236581 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:50:11.236581 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 366/744] 10.222.0.39 () {60 vars in 1140 bytes} [Fri Oct 14 02:50:10 2022] POST /identity/v3/projects => generated 262 bytes in 253 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:50:11.236581 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca3900f-92b0-4895-982f-97f50a1c38be None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:11.241302 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:11.257783 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 379/745] 10.222.0.39 () {60 vars in 1141 bytes} [Fri Oct 14 02:50:10 2022] POST /identity/v3/services => generated 242 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 02:50:11.345511 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca3900f-92b0-4895-982f-97f50a1c38be None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:11.394505 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30037146-98f8-4a20-b427-0cc22e56c4a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:11.395151 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30037146-98f8-4a20-b427-0cc22e56c4a2 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:11.395432 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30037146-98f8-4a20-b427-0cc22e56c4a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:11.395703 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30037146-98f8-4a20-b427-0cc22e56c4a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:11.403507 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30037146-98f8-4a20-b427-0cc22e56c4a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:11.405105 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30037146-98f8-4a20-b427-0cc22e56c4a2 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:11.418025 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-30037146-98f8-4a20-b427-0cc22e56c4a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:50:11.418025 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:50:11.418025 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:50:11.418025 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:50:11.418025 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:50:11.418025 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:50:11.418025 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:50:11.418025 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:50:11.418025 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:50:11.418025 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:50:11.418025 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:50:11.418025 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:50:11.418025 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 14 02:50:11.418025 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 14 02:50:11.418025 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 14 02:50:11.418984 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 14 02:50:11.418984 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:50:11.418984 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:50:11.418984 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:50:11.418984 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:50:11.418984 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:50:11.418984 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:50:11.418984 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:50:11.418984 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:50:11.418984 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:50:11.418984 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:50:11.418984 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:50:11.418984 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:50:11.418984 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:50:11.418984 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:50:11.418984 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:50:11.418984 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:50:11.419902 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 14 02:50:11.419902 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 14 02:50:11.419902 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 14 02:50:11.419902 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 14 02:50:11.419902 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 14 02:50:11.419902 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 14 02:50:11.419902 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:50:11.419902 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:50:11.419902 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 367/746] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Oct 14 02:50:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 02:50:11.425980 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8657983c-4b17-4366-8b6f-a265380b1fb7 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:11.533591 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8657983c-4b17-4366-8b6f-a265380b1fb7 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:11.570982 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322cbcf1-1d11-468a-bdff-1fabf543d0e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:11.571993 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322cbcf1-1d11-468a-bdff-1fabf543d0e2 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:11.572501 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322cbcf1-1d11-468a-bdff-1fabf543d0e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:11.572974 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322cbcf1-1d11-468a-bdff-1fabf543d0e2 None admin] PATH_INFO: `/v3/projects` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:11.573918 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322cbcf1-1d11-468a-bdff-1fabf543d0e2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:50:11.574457 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322cbcf1-1d11-468a-bdff-1fabf543d0e2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:11.576174 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322cbcf1-1d11-468a-bdff-1fabf543d0e2 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:11.597417 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 380/747] 10.222.0.39 () {56 vars in 1112 bytes} [Fri Oct 14 02:50:11 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:50:11.698588 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a99fc42-5a74-4ca7-b7a0-86dcecb55217 None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:11.698915 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a99fc42-5a74-4ca7-b7a0-86dcecb55217 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:11.699149 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a99fc42-5a74-4ca7-b7a0-86dcecb55217 None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:11.700793 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 368/748] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:50:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:50:11.707172 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52a7ae4-1914-4674-954b-701a9d47bbe2 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:11.707454 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52a7ae4-1914-4674-954b-701a9d47bbe2 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:11.707728 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52a7ae4-1914-4674-954b-701a9d47bbe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:11.727177 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-f52a7ae4-1914-4674-954b-701a9d47bbe2 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:50:11.728412 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-f52a7ae4-1914-4674-954b-701a9d47bbe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:50:11.731189 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:11.802649 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 381/749] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:50:11 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 14 02:50:12.072818 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a99fc42-5a74-4ca7-b7a0-86dcecb55217 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:12.132440 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:12.174116 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a99fc42-5a74-4ca7-b7a0-86dcecb55217 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:12.212162 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b997edee-a3a9-4034-8a98-2e501c4d5b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:12.212800 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b997edee-a3a9-4034-8a98-2e501c4d5b9b None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:12.213080 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b997edee-a3a9-4034-8a98-2e501c4d5b9b None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:12.213349 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b997edee-a3a9-4034-8a98-2e501c4d5b9b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:12.213967 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b997edee-a3a9-4034-8a98-2e501c4d5b9b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:12.215330 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b997edee-a3a9-4034-8a98-2e501c4d5b9b None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:12.224948 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-b997edee-a3a9-4034-8a98-2e501c4d5b9b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 14 02:50:12.224948 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:50:12.224948 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 14 02:50:12.224948 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 14 02:50:12.224948 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 14 02:50:12.224948 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 14 02:50:12.224948 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 14 02:50:12.224948 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:50:12.224948 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 14 02:50:12.224948 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:50:12.224948 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 02:50:12.224948 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:50:12.224948 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:50:12.224948 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:50:12.224948 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:50:12.224948 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:50:12.224948 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:50:12.226058 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:50:12.226058 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:50:12.226058 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:50:12.226058 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:50:12.226058 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:50:12.226058 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:50:12.226058 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 14 02:50:12.226058 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 14 02:50:12.226058 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 14 02:50:12.226058 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 14 02:50:12.226058 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:50:12.226058 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:50:12.226058 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:50:12.226058 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:50:12.226058 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:50:12.226058 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:50:12.226981 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:50:12.226981 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:50:12.226981 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:50:12.226981 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:50:12.226981 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:50:12.226981 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:50:12.226981 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:50:12.226981 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:50:12.226981 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:50:12.226981 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:50:12.226981 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 14 02:50:12.226981 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:50:12.226981 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 14 02:50:12.226981 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:50:12.231606 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 369/750] 10.222.0.39 () {56 vars in 1126 bytes} [Fri Oct 14 02:50:12 2022] GET /identity/v3/services/object-store => generated 93 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:50:12.233134 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52a7ae4-1914-4674-954b-701a9d47bbe2 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:12.334301 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52a7ae4-1914-4674-954b-701a9d47bbe2 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:12.372958 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44f530e-5d23-4530-9925-5c9efd10cb13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:12.373560 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44f530e-5d23-4530-9925-5c9efd10cb13 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:12.373864 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44f530e-5d23-4530-9925-5c9efd10cb13 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:12.374139 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44f530e-5d23-4530-9925-5c9efd10cb13 None admin] PATH_INFO: `/v3/services` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:12.374763 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44f530e-5d23-4530-9925-5c9efd10cb13 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:50:12.375064 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44f530e-5d23-4530-9925-5c9efd10cb13 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:12.376429 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44f530e-5d23-4530-9925-5c9efd10cb13 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:12.381499 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 382/751] 10.222.0.39 () {56 vars in 1122 bytes} [Fri Oct 14 02:50:12 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:50:12.390814 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b997edee-a3a9-4034-8a98-2e501c4d5b9b None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:12.451468 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f959296-611e-4a68-9512-293b81db90e9 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:12.452118 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f959296-611e-4a68-9512-293b81db90e9 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:12.452588 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f959296-611e-4a68-9512-293b81db90e9 None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:12.454434 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 383/752] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:50:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 14 02:50:12.460523 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5344df9-48c3-45cc-92a2-01ea55546836 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:12.460813 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5344df9-48c3-45cc-92a2-01ea55546836 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:12.461091 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5344df9-48c3-45cc-92a2-01ea55546836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:12.478112 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:12.480464 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-f5344df9-48c3-45cc-92a2-01ea55546836 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:50:12.481675 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-f5344df9-48c3-45cc-92a2-01ea55546836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:50:12.506811 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b997edee-a3a9-4034-8a98-2e501c4d5b9b None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:12.549410 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2247b12f-3589-4cc2-a30f-64d2e3bf4c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:12.550422 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2247b12f-3589-4cc2-a30f-64d2e3bf4c35 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:12.550768 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2247b12f-3589-4cc2-a30f-64d2e3bf4c35 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:12.551107 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2247b12f-3589-4cc2-a30f-64d2e3bf4c35 None admin] PATH_INFO: `/v3/services` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:12.551804 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2247b12f-3589-4cc2-a30f-64d2e3bf4c35 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:50:12.552172 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2247b12f-3589-4cc2-a30f-64d2e3bf4c35 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:12.553698 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2247b12f-3589-4cc2-a30f-64d2e3bf4c35 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:12.558105 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 384/753] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:50:12 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:50:12.562175 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:12.563784 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 370/754] 10.222.0.39 () {56 vars in 1122 bytes} [Fri Oct 14 02:50:12 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:50:12.565203 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5344df9-48c3-45cc-92a2-01ea55546836 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:12.668941 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5344df9-48c3-45cc-92a2-01ea55546836 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:12.698763 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:12.703762 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a331bc-ce7e-49b7-bb8e-3bbf6db690c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:12.704365 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a331bc-ce7e-49b7-bb8e-3bbf6db690c0 None admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:12.704642 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a331bc-ce7e-49b7-bb8e-3bbf6db690c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:12.704911 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a331bc-ce7e-49b7-bb8e-3bbf6db690c0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:12.705562 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a331bc-ce7e-49b7-bb8e-3bbf6db690c0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:12.708345 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a331bc-ce7e-49b7-bb8e-3bbf6db690c0 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:12.797691 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 385/755] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 02:50:12 2022] POST /identity/v3/endpoints => generated 325 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:50:12.835597 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2247b12f-3589-4cc2-a30f-64d2e3bf4c35 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:12.944077 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2247b12f-3589-4cc2-a30f-64d2e3bf4c35 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:12.988042 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e00b409-b7c3-460a-9e5b-723e71554760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:12.988669 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e00b409-b7c3-460a-9e5b-723e71554760 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:12.988941 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e00b409-b7c3-460a-9e5b-723e71554760 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:12.989211 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e00b409-b7c3-460a-9e5b-723e71554760 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:12.989830 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e00b409-b7c3-460a-9e5b-723e71554760 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:12.991215 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e00b409-b7c3-460a-9e5b-723e71554760 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:13.000201 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-5e00b409-b7c3-460a-9e5b-723e71554760 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 14 02:50:13.000201 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:50:13.000201 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 14 02:50:13.000201 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 14 02:50:13.000201 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 14 02:50:13.000201 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 14 02:50:13.000201 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 14 02:50:13.000201 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:50:13.000201 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 14 02:50:13.000201 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:50:13.000201 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 02:50:13.000201 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:50:13.000201 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:50:13.000201 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:50:13.000201 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:50:13.000201 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:50:13.000201 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:50:13.001250 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:50:13.001250 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:50:13.001250 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:50:13.001250 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:50:13.001250 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:50:13.001250 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:50:13.001250 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 14 02:50:13.001250 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 14 02:50:13.001250 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 14 02:50:13.001250 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 14 02:50:13.001250 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:50:13.001250 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:50:13.001250 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:50:13.001250 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:50:13.001250 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:50:13.001250 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:50:13.002142 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:50:13.002142 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:50:13.002142 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:50:13.002142 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:50:13.002142 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:50:13.002142 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:50:13.002142 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:50:13.002142 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:50:13.002142 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:50:13.002142 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:50:13.002142 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 14 02:50:13.002142 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:50:13.002142 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 14 02:50:13.002142 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:50:13.005365 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 371/756] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Oct 14 02:50:12 2022] GET /identity/v3/services/compute => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:50:13.010280 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a331bc-ce7e-49b7-bb8e-3bbf6db690c0 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:13.118715 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a331bc-ce7e-49b7-bb8e-3bbf6db690c0 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:13.153638 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9690fbef-18f2-4ccf-a658-ba087b8dc3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:13.154278 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9690fbef-18f2-4ccf-a658-ba087b8dc3ca None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:13.154563 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9690fbef-18f2-4ccf-a658-ba087b8dc3ca None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:13.154826 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9690fbef-18f2-4ccf-a658-ba087b8dc3ca None admin] PATH_INFO: `/v3/services` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:13.155458 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9690fbef-18f2-4ccf-a658-ba087b8dc3ca None admin] RBAC: Adding query filter params (name=compute) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:50:13.155765 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9690fbef-18f2-4ccf-a658-ba087b8dc3ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:13.157153 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9690fbef-18f2-4ccf-a658-ba087b8dc3ca None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:13.162367 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 386/757] 10.222.0.39 () {56 vars in 1112 bytes} [Fri Oct 14 02:50:13 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:50:13.168045 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e00b409-b7c3-460a-9e5b-723e71554760 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:13.217697 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:13.271727 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e00b409-b7c3-460a-9e5b-723e71554760 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:13.305617 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e9dc39-572b-49e0-8a4c-268f616a860a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:13.306247 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e9dc39-572b-49e0-8a4c-268f616a860a None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:13.306525 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e9dc39-572b-49e0-8a4c-268f616a860a None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:13.306794 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e9dc39-572b-49e0-8a4c-268f616a860a None admin] PATH_INFO: `/v3/services` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:13.307418 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e9dc39-572b-49e0-8a4c-268f616a860a None admin] RBAC: Adding query filter params (type=compute) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:50:13.307717 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e9dc39-572b-49e0-8a4c-268f616a860a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:13.309074 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e9dc39-572b-49e0-8a4c-268f616a860a None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:13.314308 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 372/758] 10.222.0.39 () {56 vars in 1112 bytes} [Fri Oct 14 02:50:13 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:50:13.323106 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9690fbef-18f2-4ccf-a658-ba087b8dc3ca None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:13.380960 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:13.432765 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9690fbef-18f2-4ccf-a658-ba087b8dc3ca None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:13.472827 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b9e76d-3046-4f92-89f9-a37b65a4c312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:13.473671 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b9e76d-3046-4f92-89f9-a37b65a4c312 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:13.474174 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b9e76d-3046-4f92-89f9-a37b65a4c312 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:13.474603 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b9e76d-3046-4f92-89f9-a37b65a4c312 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:13.475455 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b9e76d-3046-4f92-89f9-a37b65a4c312 None admin] RBAC: Adding query filter params (interface=public, service_id=afa041f0b2c741869db2696fb1a97b16, region_id=RegionOne) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:50:13.475915 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b9e76d-3046-4f92-89f9-a37b65a4c312 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:13.477436 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b9e76d-3046-4f92-89f9-a37b65a4c312 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:13.483552 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 387/759] 10.222.0.39 () {56 vars in 1251 bytes} [Fri Oct 14 02:50:13 2022] GET /identity/v3/endpoints?service_id=afa041f0b2c741869db2696fb1a97b16&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:50:13.493793 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e9dc39-572b-49e0-8a4c-268f616a860a None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:13.544955 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0c7c10-41e9-4034-ad12-fcd4890f565d None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:13.545244 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0c7c10-41e9-4034-ad12-fcd4890f565d None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:13.545518 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0c7c10-41e9-4034-ad12-fcd4890f565d None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:13.547150 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 388/760] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:50:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:50:13.552846 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5726f1-1fe1-47db-abff-bb145a1f781b None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:13.553132 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5726f1-1fe1-47db-abff-bb145a1f781b None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:13.553560 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5726f1-1fe1-47db-abff-bb145a1f781b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:13.575792 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-ec5726f1-1fe1-47db-abff-bb145a1f781b None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:50:13.577283 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-ec5726f1-1fe1-47db-abff-bb145a1f781b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:50:13.599828 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e9dc39-572b-49e0-8a4c-268f616a860a None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:13.611754 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:13.637782 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b4f6ea-5c63-44e5-9c68-58b3f61ab863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:13.638400 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b4f6ea-5c63-44e5-9c68-58b3f61ab863 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:13.638695 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b4f6ea-5c63-44e5-9c68-58b3f61ab863 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:13.638956 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b4f6ea-5c63-44e5-9c68-58b3f61ab863 None admin] PATH_INFO: `/v3/services` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:13.639915 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b4f6ea-5c63-44e5-9c68-58b3f61ab863 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:13.641872 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b4f6ea-5c63-44e5-9c68-58b3f61ab863 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:13.653811 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 373/761] 10.222.0.39 () {56 vars in 1087 bytes} [Fri Oct 14 02:50:13 2022] GET /identity/v3/services => generated 1957 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:50:13.664200 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 389/762] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:50:13 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 14 02:50:13.937004 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b4f6ea-5c63-44e5-9c68-58b3f61ab863 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:14.048227 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b4f6ea-5c63-44e5-9c68-58b3f61ab863 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:14.062216 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85da42fb-b29a-4c83-9689-b339dffeab40 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:14.062720 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85da42fb-b29a-4c83-9689-b339dffeab40 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:14.062831 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85da42fb-b29a-4c83-9689-b339dffeab40 None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:14.064360 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 390/763] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:50:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 14 02:50:14.069974 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bea616-a334-4931-99ce-7921b385ce3e None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:14.070264 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bea616-a334-4931-99ce-7921b385ce3e None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:14.070544 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bea616-a334-4931-99ce-7921b385ce3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:14.089658 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-c1bea616-a334-4931-99ce-7921b385ce3e None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:50:14.091105 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fc44ec-515c-4657-b9a4-9bb7ec4c09f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:14.091364 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-c1bea616-a334-4931-99ce-7921b385ce3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:50:14.092077 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc44ec-515c-4657-b9a4-9bb7ec4c09f5 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:14.092502 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc44ec-515c-4657-b9a4-9bb7ec4c09f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:14.092935 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc44ec-515c-4657-b9a4-9bb7ec4c09f5 None admin] PATH_INFO: `/v3/services/image` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:14.093886 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fc44ec-515c-4657-b9a4-9bb7ec4c09f5 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:14.096315 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fc44ec-515c-4657-b9a4-9bb7ec4c09f5 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:14.107164 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-17fc44ec-515c-4657-b9a4-9bb7ec4c09f5 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 14 02:50:14.107164 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:50:14.107164 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 14 02:50:14.107164 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 14 02:50:14.107164 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 14 02:50:14.107164 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 14 02:50:14.107164 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 14 02:50:14.107164 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:50:14.107164 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 14 02:50:14.107164 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:50:14.107164 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 02:50:14.107164 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:50:14.107164 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:50:14.107164 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:50:14.107164 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:50:14.107164 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:50:14.107164 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:50:14.108421 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:50:14.108421 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:50:14.108421 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:50:14.108421 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:50:14.108421 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:50:14.108421 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:50:14.108421 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 14 02:50:14.108421 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 14 02:50:14.108421 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 14 02:50:14.108421 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 14 02:50:14.108421 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:50:14.108421 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:50:14.108421 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:50:14.108421 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:50:14.108421 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:50:14.108421 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:50:14.109397 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:50:14.109397 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:50:14.109397 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:50:14.109397 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:50:14.109397 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:50:14.109397 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:50:14.109397 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:50:14.109397 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:50:14.109397 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:50:14.109397 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:50:14.109397 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 14 02:50:14.109397 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:50:14.109397 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 14 02:50:14.109397 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:50:14.111807 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 374/764] 10.222.0.39 () {56 vars in 1105 bytes} [Fri Oct 14 02:50:13 2022] GET /identity/v3/services/image => generated 86 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:50:14.118266 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fc44ec-515c-4657-b9a4-9bb7ec4c09f5 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:14.154162 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:14.165059 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:14.189890 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 391/765] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:50:14 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 14 02:50:14.236321 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fc44ec-515c-4657-b9a4-9bb7ec4c09f5 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:14.272112 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb2a9b7-c153-47ce-b8cf-0eba1b5f67a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:14.272920 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb2a9b7-c153-47ce-b8cf-0eba1b5f67a7 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:14.273354 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb2a9b7-c153-47ce-b8cf-0eba1b5f67a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:14.273869 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb2a9b7-c153-47ce-b8cf-0eba1b5f67a7 None admin] PATH_INFO: `/v3/services` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:14.274720 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb2a9b7-c153-47ce-b8cf-0eba1b5f67a7 None admin] RBAC: Adding query filter params (name=image) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:50:14.275179 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb2a9b7-c153-47ce-b8cf-0eba1b5f67a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:14.276762 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb2a9b7-c153-47ce-b8cf-0eba1b5f67a7 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:14.282170 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 375/766] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:50:14 2022] GET /identity/v3/services?name=image => generated 122 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:50:14.290004 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bea616-a334-4931-99ce-7921b385ce3e None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:14.397714 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bea616-a334-4931-99ce-7921b385ce3e None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:14.432847 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8138f6-5cfc-4fb4-94f7-3a2e57d418b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:14.433449 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8138f6-5cfc-4fb4-94f7-3a2e57d418b1 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:14.433729 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8138f6-5cfc-4fb4-94f7-3a2e57d418b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:14.434031 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8138f6-5cfc-4fb4-94f7-3a2e57d418b1 None admin] PATH_INFO: `/v3/services` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:14.434660 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8138f6-5cfc-4fb4-94f7-3a2e57d418b1 None admin] RBAC: Adding query filter params (type=image) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:50:14.434962 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8138f6-5cfc-4fb4-94f7-3a2e57d418b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:14.436335 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8138f6-5cfc-4fb4-94f7-3a2e57d418b1 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:14.441710 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 392/767] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:50:14 2022] GET /identity/v3/services?type=image => generated 350 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:50:14.447533 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb2a9b7-c153-47ce-b8cf-0eba1b5f67a7 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:14.478657 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8138f6-5cfc-4fb4-94f7-3a2e57d418b1 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:14.538591 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:14.568909 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb2a9b7-c153-47ce-b8cf-0eba1b5f67a7 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:14.571413 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:14.612857 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8138f6-5cfc-4fb4-94f7-3a2e57d418b1 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:14.613415 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c22909-d86f-4fa7-aa64-12b5ee4349c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:14.614062 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c22909-d86f-4fa7-aa64-12b5ee4349c0 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:14.614344 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c22909-d86f-4fa7-aa64-12b5ee4349c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:14.614621 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c22909-d86f-4fa7-aa64-12b5ee4349c0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:14.615321 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c22909-d86f-4fa7-aa64-12b5ee4349c0 None admin] RBAC: Adding query filter params (interface=public, service_id=f5af7962ddbe4ccb850e4ab5659fc2c4, region_id=RegionOne) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:50:14.615874 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c22909-d86f-4fa7-aa64-12b5ee4349c0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:14.618224 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c22909-d86f-4fa7-aa64-12b5ee4349c0 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:14.625977 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 376/768] 10.222.0.39 () {56 vars in 1251 bytes} [Fri Oct 14 02:50:14 2022] GET /identity/v3/endpoints?service_id=f5af7962ddbe4ccb850e4ab5659fc2c4&interface=public®ion_id=RegionOne => generated 194 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:50:14.629445 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c22909-d86f-4fa7-aa64-12b5ee4349c0 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:14.661383 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f8b8fb-44a1-45cc-8c6a-4d4e556558f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:14.662017 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f8b8fb-44a1-45cc-8c6a-4d4e556558f3 None admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:14.662299 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f8b8fb-44a1-45cc-8c6a-4d4e556558f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:14.662575 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f8b8fb-44a1-45cc-8c6a-4d4e556558f3 None admin] PATH_INFO: `/v3/users` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:14.663521 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f8b8fb-44a1-45cc-8c6a-4d4e556558f3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:14.665209 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f8b8fb-44a1-45cc-8c6a-4d4e556558f3 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:14.668746 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: WARNING keystone.server.flask.common [None req-02f8b8fb-44a1-45cc-8c6a-4d4e556558f3 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 14 02:50:14.708633 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.sql.core [None req-02f8b8fb-44a1-45cc-8c6a-4d4e556558f3 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 14 02:50:14.708633 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 14 02:50:14.708633 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [parameters: {'user_id': '369437d2d20d474a8f9a728d72fa85a2', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 14 02:50:14.708633 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81517) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 14 02:50:14.734709 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-02f8b8fb-44a1-45cc-8c6a-4d4e556558f3 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 14 02:50:14.734709 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:50:14.734709 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 14 02:50:14.734709 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 14 02:50:14.734709 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 14 02:50:14.734709 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 14 02:50:14.734709 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 14 02:50:14.734709 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application result = self._query(query) Oct 14 02:50:14.734709 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 14 02:50:14.734709 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application conn.query(q) Oct 14 02:50:14.734709 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 14 02:50:14.734709 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 14 02:50:14.734709 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 14 02:50:14.734709 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application result.read() Oct 14 02:50:14.734709 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 14 02:50:14.735783 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 14 02:50:14.735783 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 14 02:50:14.735783 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 14 02:50:14.735783 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 14 02:50:14.735783 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 14 02:50:14.735783 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 14 02:50:14.735783 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 14 02:50:14.735783 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 14 02:50:14.735783 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:50:14.735783 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 14 02:50:14.735783 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:50:14.735783 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:50:14.735783 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 14 02:50:14.735783 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 14 02:50:14.735783 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Oct 14 02:50:14.735783 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Oct 14 02:50:14.735783 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 14 02:50:14.735783 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application next(self.gen) Oct 14 02:50:14.736722 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 14 02:50:14.736722 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application yield resource Oct 14 02:50:14.736722 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 14 02:50:14.736722 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application next(self.gen) Oct 14 02:50:14.736722 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 14 02:50:14.736722 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application self.session.rollback() Oct 14 02:50:14.736722 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 14 02:50:14.736722 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application self.force_reraise() Oct 14 02:50:14.736722 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 14 02:50:14.736722 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise self.value Oct 14 02:50:14.736722 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 14 02:50:14.736722 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 14 02:50:14.736722 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 14 02:50:14.736722 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application session.commit() Oct 14 02:50:14.736722 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 14 02:50:14.736722 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 14 02:50:14.736722 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 14 02:50:14.738184 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application self._prepare_impl() Oct 14 02:50:14.738184 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 14 02:50:14.738184 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application self.session.flush() Oct 14 02:50:14.738184 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 14 02:50:14.738184 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application self._flush(objects) Oct 14 02:50:14.738184 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 14 02:50:14.738184 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 14 02:50:14.738184 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 14 02:50:14.738184 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application compat.raise_( Oct 14 02:50:14.738184 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 14 02:50:14.738184 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception Oct 14 02:50:14.738184 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 14 02:50:14.738184 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application flush_context.execute() Oct 14 02:50:14.738184 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 14 02:50:14.738184 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rec.execute(self) Oct 14 02:50:14.738184 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 14 02:50:14.738184 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 14 02:50:14.740414 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 14 02:50:14.740414 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 14 02:50:14.740414 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Oct 14 02:50:14.740414 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application result = connection._execute_20( Oct 14 02:50:14.740414 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 14 02:50:14.740414 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 14 02:50:14.740414 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 14 02:50:14.740414 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 14 02:50:14.740414 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 14 02:50:14.740414 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 14 02:50:14.740414 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 14 02:50:14.740414 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 14 02:50:14.740414 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 14 02:50:14.740414 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 14 02:50:14.740414 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 14 02:50:14.740414 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception Oct 14 02:50:14.742518 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 14 02:50:14.742518 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 14 02:50:14.742518 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 14 02:50:14.742518 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 14 02:50:14.742518 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 14 02:50:14.742518 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application result = self._query(query) Oct 14 02:50:14.742518 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 14 02:50:14.742518 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application conn.query(q) Oct 14 02:50:14.742518 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 14 02:50:14.742518 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 14 02:50:14.742518 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 14 02:50:14.742518 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application result.read() Oct 14 02:50:14.742518 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 14 02:50:14.742518 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 14 02:50:14.742518 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 14 02:50:14.742518 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 14 02:50:14.744470 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 14 02:50:14.744470 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 14 02:50:14.744470 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 14 02:50:14.744470 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 14 02:50:14.744470 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 14 02:50:14.744470 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 14 02:50:14.744470 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [parameters: {'user_id': '369437d2d20d474a8f9a728d72fa85a2', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 14 02:50:14.744470 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 14 02:50:14.744470 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:50:14.744470 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 02:50:14.744470 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:50:14.744470 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:50:14.744470 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:50:14.744470 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:50:14.744470 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:50:14.746366 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:50:14.746366 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:50:14.746366 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:50:14.746366 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:50:14.746366 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:50:14.746366 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:50:14.746366 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:50:14.746366 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Oct 14 02:50:14.746366 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Oct 14 02:50:14.746366 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:50:14.746366 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:50:14.746366 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 14 02:50:14.746366 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:50:14.746366 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 14 02:50:14.746366 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:50:14.746366 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Oct 14 02:50:14.746366 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Oct 14 02:50:14.748950 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Oct 14 02:50:14.748950 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Oct 14 02:50:14.748950 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 14 02:50:14.748950 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 14 02:50:14.748950 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 14 02:50:14.748950 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:50:14.748950 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 393/769] 10.222.0.39 () {60 vars in 1131 bytes} [Fri Oct 14 02:50:14 2022] POST /identity/v3/users => generated 160 bytes in 263 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:50:14.748950 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f8b8fb-44a1-45cc-8c6a-4d4e556558f3 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:14.755358 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c22909-d86f-4fa7-aa64-12b5ee4349c0 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:14.792289 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e04239-bdfc-4828-87df-a766c4f1abf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:14.793272 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e04239-bdfc-4828-87df-a766c4f1abf9 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:14.793795 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e04239-bdfc-4828-87df-a766c4f1abf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:14.794280 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e04239-bdfc-4828-87df-a766c4f1abf9 None admin] PATH_INFO: `/v3/services` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:14.795094 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e04239-bdfc-4828-87df-a766c4f1abf9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:14.796725 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e04239-bdfc-4828-87df-a766c4f1abf9 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:14.804339 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 377/770] 10.222.0.39 () {56 vars in 1087 bytes} [Fri Oct 14 02:50:14 2022] GET /identity/v3/services => generated 1957 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 02:50:14.874582 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f8b8fb-44a1-45cc-8c6a-4d4e556558f3 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:14.883473 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:14.914702 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcecd78-e3db-45a4-81ff-88ff9eb5220a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:14.915503 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcecd78-e3db-45a4-81ff-88ff9eb5220a None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:14.915886 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcecd78-e3db-45a4-81ff-88ff9eb5220a None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:14.916261 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcecd78-e3db-45a4-81ff-88ff9eb5220a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:14.925435 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcecd78-e3db-45a4-81ff-88ff9eb5220a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:14.927545 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcecd78-e3db-45a4-81ff-88ff9eb5220a None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:14.940677 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-2fcecd78-e3db-45a4-81ff-88ff9eb5220a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 14 02:50:14.940677 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:50:14.940677 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:50:14.940677 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:50:14.940677 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:50:14.940677 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:50:14.940677 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:50:14.940677 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:50:14.940677 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:50:14.940677 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:50:14.940677 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:50:14.940677 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:50:14.940677 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 14 02:50:14.940677 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 14 02:50:14.940677 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 14 02:50:14.940677 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 14 02:50:14.941798 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:50:14.941798 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:50:14.941798 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 14 02:50:14.941798 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:50:14.941798 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 14 02:50:14.941798 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:50:14.941798 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:50:14.941798 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:50:14.941798 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:50:14.941798 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:50:14.941798 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:50:14.941798 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:50:14.941798 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:50:14.941798 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:50:14.941798 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:50:14.941798 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:50:14.941798 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:50:14.941798 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:50:14.942753 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:50:14.942753 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:50:14.942753 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 14 02:50:14.942753 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 14 02:50:14.942753 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 14 02:50:14.942753 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 14 02:50:14.942753 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 14 02:50:14.942753 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 14 02:50:14.942753 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 14 02:50:14.942753 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:50:14.942753 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 394/771] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Oct 14 02:50:14 2022] GET /identity/v3/users/cinder => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:50:14.950393 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e04239-bdfc-4828-87df-a766c4f1abf9 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:14.990735 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:15.066366 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e04239-bdfc-4828-87df-a766c4f1abf9 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:15.091999 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9095513a-c246-4ec5-9ce7-0fe34e15d206 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:15.092505 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9095513a-c246-4ec5-9ce7-0fe34e15d206 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:15.092582 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9095513a-c246-4ec5-9ce7-0fe34e15d206 None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:15.097593 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 395/772] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:50:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:50:15.102893 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6b4eb3-c0d5-479f-8a83-feec835e3c1d None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:15.103185 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6b4eb3-c0d5-479f-8a83-feec835e3c1d None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:15.103465 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6b4eb3-c0d5-479f-8a83-feec835e3c1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:15.106717 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b599d222-696f-4286-8f83-79489005a028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:15.107575 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b599d222-696f-4286-8f83-79489005a028 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:15.108050 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b599d222-696f-4286-8f83-79489005a028 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:15.108534 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b599d222-696f-4286-8f83-79489005a028 None admin] PATH_INFO: `/v3/users` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:15.109414 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b599d222-696f-4286-8f83-79489005a028 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:50:15.109941 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b599d222-696f-4286-8f83-79489005a028 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:15.111545 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b599d222-696f-4286-8f83-79489005a028 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:15.123191 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-0d6b4eb3-c0d5-479f-8a83-feec835e3c1d None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:50:15.124790 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-0d6b4eb3-c0d5-479f-8a83-feec835e3c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:50:15.147952 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 378/773] 10.222.0.39 () {56 vars in 1101 bytes} [Fri Oct 14 02:50:14 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:50:15.204416 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 396/774] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:50:15 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 14 02:50:15.478270 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b599d222-696f-4286-8f83-79489005a028 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:15.566103 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:15.578230 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b599d222-696f-4286-8f83-79489005a028 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:15.611574 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e7a1cb-8ca5-48f4-8e85-939c35974966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:15.612795 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e7a1cb-8ca5-48f4-8e85-939c35974966 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:15.613080 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e7a1cb-8ca5-48f4-8e85-939c35974966 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:15.613352 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e7a1cb-8ca5-48f4-8e85-939c35974966 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:15.614473 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e7a1cb-8ca5-48f4-8e85-939c35974966 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:15.616163 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e7a1cb-8ca5-48f4-8e85-939c35974966 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:15.621897 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 379/775] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:50:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:50:15.623543 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6b4eb3-c0d5-479f-8a83-feec835e3c1d None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:15.719023 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:15.724856 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6b4eb3-c0d5-479f-8a83-feec835e3c1d None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:15.765085 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817ac902-2270-4e88-ba73-22722d8047a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:15.765625 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817ac902-2270-4e88-ba73-22722d8047a9 None admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:15.765932 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817ac902-2270-4e88-ba73-22722d8047a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:15.766210 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817ac902-2270-4e88-ba73-22722d8047a9 None admin] PATH_INFO: `/v3/projects` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:15.766888 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817ac902-2270-4e88-ba73-22722d8047a9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:15.768414 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817ac902-2270-4e88-ba73-22722d8047a9 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:15.856679 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cdab51-993d-405b-a754-38a64e5f12e6 None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:15.856966 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cdab51-993d-405b-a754-38a64e5f12e6 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:15.857251 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cdab51-993d-405b-a754-38a64e5f12e6 None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:15.858626 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 380/776] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:50:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:50:15.864131 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437998e7-1e03-451a-b28f-d9afdf4569df None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:15.864411 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437998e7-1e03-451a-b28f-d9afdf4569df None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:15.864684 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437998e7-1e03-451a-b28f-d9afdf4569df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:15.884137 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-437998e7-1e03-451a-b28f-d9afdf4569df None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:50:15.886221 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-437998e7-1e03-451a-b28f-d9afdf4569df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:50:15.889639 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 397/777] 10.222.0.39 () {60 vars in 1141 bytes} [Fri Oct 14 02:50:15 2022] POST /identity/v3/projects => generated 311 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:50:15.970708 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 381/778] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:50:15 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 14 02:50:16.239013 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817ac902-2270-4e88-ba73-22722d8047a9 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:16.338759 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817ac902-2270-4e88-ba73-22722d8047a9 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:16.371660 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1534533d-bedc-4d2b-bc9c-721d35f6e963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:16.372285 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1534533d-bedc-4d2b-bc9c-721d35f6e963 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:16.372567 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1534533d-bedc-4d2b-bc9c-721d35f6e963 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:16.372838 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1534533d-bedc-4d2b-bc9c-721d35f6e963 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:16.373419 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1534533d-bedc-4d2b-bc9c-721d35f6e963 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:16.374809 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1534533d-bedc-4d2b-bc9c-721d35f6e963 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:16.383497 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-1534533d-bedc-4d2b-bc9c-721d35f6e963 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 14 02:50:16.383497 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:50:16.383497 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 14 02:50:16.383497 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 14 02:50:16.383497 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 14 02:50:16.383497 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 14 02:50:16.383497 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 14 02:50:16.383497 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:50:16.383497 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 14 02:50:16.383497 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:50:16.383497 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 02:50:16.383497 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:50:16.383497 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:50:16.383497 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:50:16.383497 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:50:16.383497 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:50:16.383497 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:50:16.384474 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:50:16.384474 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:50:16.384474 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:50:16.384474 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:50:16.384474 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:50:16.384474 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:50:16.384474 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 14 02:50:16.384474 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 14 02:50:16.384474 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 14 02:50:16.384474 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 14 02:50:16.384474 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:50:16.384474 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:50:16.384474 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:50:16.384474 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:50:16.384474 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:50:16.384474 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:50:16.385327 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:50:16.385327 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:50:16.385327 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:50:16.385327 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:50:16.385327 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:50:16.385327 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:50:16.385327 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:50:16.385327 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:50:16.385327 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:50:16.385327 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:50:16.385327 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 14 02:50:16.385327 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:50:16.385327 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 14 02:50:16.385327 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:50:16.389560 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 398/779] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Oct 14 02:50:16 2022] GET /identity/v3/services/compute => generated 88 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:50:16.390582 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437998e7-1e03-451a-b28f-d9afdf4569df None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:16.478006 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:16.495684 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437998e7-1e03-451a-b28f-d9afdf4569df None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:16.527209 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b82c4c-efcd-4acd-9b22-703cb902e946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:16.527817 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b82c4c-efcd-4acd-9b22-703cb902e946 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:16.528094 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b82c4c-efcd-4acd-9b22-703cb902e946 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:16.528362 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b82c4c-efcd-4acd-9b22-703cb902e946 None admin] PATH_INFO: `/v3/services` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:16.529011 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b82c4c-efcd-4acd-9b22-703cb902e946 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:50:16.529317 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b82c4c-efcd-4acd-9b22-703cb902e946 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:16.530719 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b82c4c-efcd-4acd-9b22-703cb902e946 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:16.540781 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 382/780] 10.222.0.39 () {56 vars in 1112 bytes} [Fri Oct 14 02:50:16 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:50:16.542342 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1534533d-bedc-4d2b-bc9c-721d35f6e963 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:16.558301 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:16.644679 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1534533d-bedc-4d2b-bc9c-721d35f6e963 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:16.678574 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307b0414-4349-4d4d-a7b8-d4d7f165ef4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:16.679178 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307b0414-4349-4d4d-a7b8-d4d7f165ef4c None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:16.679458 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307b0414-4349-4d4d-a7b8-d4d7f165ef4c None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:16.679731 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307b0414-4349-4d4d-a7b8-d4d7f165ef4c None admin] PATH_INFO: `/v3/services` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:16.680357 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307b0414-4349-4d4d-a7b8-d4d7f165ef4c None admin] RBAC: Adding query filter params (type=compute) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:50:16.680665 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307b0414-4349-4d4d-a7b8-d4d7f165ef4c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:16.682072 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307b0414-4349-4d4d-a7b8-d4d7f165ef4c None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:16.687139 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 399/781] 10.222.0.39 () {56 vars in 1112 bytes} [Fri Oct 14 02:50:16 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:50:16.693284 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b82c4c-efcd-4acd-9b22-703cb902e946 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:16.794194 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b82c4c-efcd-4acd-9b22-703cb902e946 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:16.829565 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cf1a56-9126-44b8-9651-af04018b041c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:16.830201 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cf1a56-9126-44b8-9651-af04018b041c None admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:16.830483 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cf1a56-9126-44b8-9651-af04018b041c None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:16.831347 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cf1a56-9126-44b8-9651-af04018b041c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:16.832023 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cf1a56-9126-44b8-9651-af04018b041c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:16.833375 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cf1a56-9126-44b8-9651-af04018b041c None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:16.949809 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 383/782] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 02:50:16 2022] POST /identity/v3/endpoints => generated 334 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:50:17.085581 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5365fb-5978-4108-8d6d-5450e6ed719b None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:17.086006 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5365fb-5978-4108-8d6d-5450e6ed719b None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:17.086167 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5365fb-5978-4108-8d6d-5450e6ed719b None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:17.087726 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 400/783] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:50:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:50:17.093276 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70b248e-a435-4792-b586-f66112a574a9 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:17.093574 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70b248e-a435-4792-b586-f66112a574a9 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:17.093890 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70b248e-a435-4792-b586-f66112a574a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:17.107343 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:17.114587 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-b70b248e-a435-4792-b586-f66112a574a9 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:50:17.115817 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-b70b248e-a435-4792-b586-f66112a574a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:50:17.191872 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 384/784] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:50:17 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:50:17.458433 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5365fb-5978-4108-8d6d-5450e6ed719b None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:17.538655 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:17.563450 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5365fb-5978-4108-8d6d-5450e6ed719b None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:17.599221 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a286154d-4e20-47ea-bb78-38af98d23dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:17.600120 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a286154d-4e20-47ea-bb78-38af98d23dae None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:17.600564 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a286154d-4e20-47ea-bb78-38af98d23dae None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:17.600984 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a286154d-4e20-47ea-bb78-38af98d23dae None admin] PATH_INFO: `/v3/services/image` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:17.601709 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a286154d-4e20-47ea-bb78-38af98d23dae None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:17.603252 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a286154d-4e20-47ea-bb78-38af98d23dae None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:17.613039 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-a286154d-4e20-47ea-bb78-38af98d23dae None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 14 02:50:17.613039 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:50:17.613039 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 14 02:50:17.613039 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 14 02:50:17.613039 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 14 02:50:17.613039 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 14 02:50:17.613039 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 14 02:50:17.613039 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:50:17.613039 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 14 02:50:17.613039 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:50:17.613039 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 02:50:17.613039 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:50:17.613039 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:50:17.613039 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:50:17.613039 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:50:17.613039 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:50:17.613039 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:50:17.614148 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:50:17.614148 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:50:17.614148 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:50:17.614148 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:50:17.614148 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:50:17.614148 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:50:17.614148 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 14 02:50:17.614148 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 14 02:50:17.614148 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 14 02:50:17.614148 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 14 02:50:17.614148 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:50:17.614148 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:50:17.614148 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:50:17.614148 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:50:17.614148 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:50:17.614148 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:50:17.615067 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:50:17.615067 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:50:17.615067 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:50:17.615067 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:50:17.615067 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:50:17.615067 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:50:17.615067 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:50:17.615067 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:50:17.615067 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:50:17.615067 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:50:17.615067 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 14 02:50:17.615067 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:50:17.615067 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 14 02:50:17.615067 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:50:17.617907 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 401/785] 10.222.0.39 () {56 vars in 1105 bytes} [Fri Oct 14 02:50:17 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:50:17.620892 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70b248e-a435-4792-b586-f66112a574a9 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:17.720254 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70b248e-a435-4792-b586-f66112a574a9 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:17.750590 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:17.756339 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87ee57d-5c09-45e4-a473-20b6c1778e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:17.756965 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87ee57d-5c09-45e4-a473-20b6c1778e74 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:17.757246 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87ee57d-5c09-45e4-a473-20b6c1778e74 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:17.757520 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87ee57d-5c09-45e4-a473-20b6c1778e74 None admin] PATH_INFO: `/v3/services` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:17.758313 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87ee57d-5c09-45e4-a473-20b6c1778e74 None admin] RBAC: Adding query filter params (name=image) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:50:17.758617 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87ee57d-5c09-45e4-a473-20b6c1778e74 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:17.759994 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87ee57d-5c09-45e4-a473-20b6c1778e74 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:17.770119 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 385/786] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:50:17 2022] GET /identity/v3/services?name=image => generated 122 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:50:17.771512 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a286154d-4e20-47ea-bb78-38af98d23dae None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:17.874896 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a286154d-4e20-47ea-bb78-38af98d23dae None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:17.908424 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0ed0f8-c388-4774-a8ad-b5fb4a26b21d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:17.909589 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0ed0f8-c388-4774-a8ad-b5fb4a26b21d None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:17.909903 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0ed0f8-c388-4774-a8ad-b5fb4a26b21d None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:17.910181 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0ed0f8-c388-4774-a8ad-b5fb4a26b21d None admin] PATH_INFO: `/v3/services` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:17.910810 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0ed0f8-c388-4774-a8ad-b5fb4a26b21d None admin] RBAC: Adding query filter params (type=image) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:50:17.911116 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0ed0f8-c388-4774-a8ad-b5fb4a26b21d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:17.912498 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0ed0f8-c388-4774-a8ad-b5fb4a26b21d None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:17.917634 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 402/787] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:50:17 2022] GET /identity/v3/services?type=image => generated 350 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:50:17.923392 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87ee57d-5c09-45e4-a473-20b6c1778e74 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:18.041897 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87ee57d-5c09-45e4-a473-20b6c1778e74 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:18.079020 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ca4e0-c277-4615-9a78-c7141877f95e None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:18.079340 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ca4e0-c277-4615-9a78-c7141877f95e None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:18.079582 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ca4e0-c277-4615-9a78-c7141877f95e None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:18.081612 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 403/788] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:50:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 14 02:50:18.085281 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc37f03-7990-45ed-b013-ccc0522cf026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:18.086524 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391c6d4d-e325-47e1-be02-b0f1c982e9b4 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:18.086808 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391c6d4d-e325-47e1-be02-b0f1c982e9b4 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:18.087082 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391c6d4d-e325-47e1-be02-b0f1c982e9b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:18.091829 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc37f03-7990-45ed-b013-ccc0522cf026 None admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:18.092125 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc37f03-7990-45ed-b013-ccc0522cf026 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:18.092410 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc37f03-7990-45ed-b013-ccc0522cf026 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:18.093084 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc37f03-7990-45ed-b013-ccc0522cf026 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:18.095294 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc37f03-7990-45ed-b013-ccc0522cf026 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:18.098316 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:18.109906 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-391c6d4d-e325-47e1-be02-b0f1c982e9b4 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:50:18.111521 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-391c6d4d-e325-47e1-be02-b0f1c982e9b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:50:18.182227 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 404/789] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:50:18 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 14 02:50:18.195500 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 386/790] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 02:50:17 2022] POST /identity/v3/endpoints => generated 327 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:50:18.452156 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391c6d4d-e325-47e1-be02-b0f1c982e9b4 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:18.552459 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391c6d4d-e325-47e1-be02-b0f1c982e9b4 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:18.586573 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:18.593204 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77127a77-8878-48e6-aeff-f52ff6249668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:18.593870 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77127a77-8878-48e6-aeff-f52ff6249668 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:18.594156 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77127a77-8878-48e6-aeff-f52ff6249668 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:18.594429 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77127a77-8878-48e6-aeff-f52ff6249668 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:18.595528 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77127a77-8878-48e6-aeff-f52ff6249668 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:18.596993 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77127a77-8878-48e6-aeff-f52ff6249668 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:18.601840 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 405/791] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:50:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:50:18.604908 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc37f03-7990-45ed-b013-ccc0522cf026 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:18.659880 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:18.708780 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc37f03-7990-45ed-b013-ccc0522cf026 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:18.743646 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16749b00-875d-49f4-bf85-68f6e0177a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:18.744261 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16749b00-875d-49f4-bf85-68f6e0177a98 None admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:18.744544 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16749b00-875d-49f4-bf85-68f6e0177a98 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:18.744819 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16749b00-875d-49f4-bf85-68f6e0177a98 None admin] PATH_INFO: `/v3/users` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:18.745481 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16749b00-875d-49f4-bf85-68f6e0177a98 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:18.746965 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16749b00-875d-49f4-bf85-68f6e0177a98 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:18.831875 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 387/792] 10.222.0.39 () {60 vars in 1132 bytes} [Fri Oct 14 02:50:18 2022] POST /identity/v3/users => generated 285 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:50:19.214730 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e704c8f-e53f-4ff9-9ce7-5828eea53911 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:19.215057 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e704c8f-e53f-4ff9-9ce7-5828eea53911 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:19.215290 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e704c8f-e53f-4ff9-9ce7-5828eea53911 None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:19.216640 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 406/793] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:50:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:50:19.222093 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d300737-db81-4c01-959f-38894e52f724 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:19.222382 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d300737-db81-4c01-959f-38894e52f724 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:19.222652 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d300737-db81-4c01-959f-38894e52f724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:19.242158 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-4d300737-db81-4c01-959f-38894e52f724 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:50:19.243350 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-4d300737-db81-4c01-959f-38894e52f724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:50:19.310451 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 388/794] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:50:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:50:19.577992 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e704c8f-e53f-4ff9-9ce7-5828eea53911 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:19.680219 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e704c8f-e53f-4ff9-9ce7-5828eea53911 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:19.713119 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0364ff1-d76c-4119-800b-c256e3ed545d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:19.713735 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0364ff1-d76c-4119-800b-c256e3ed545d None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:19.714041 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0364ff1-d76c-4119-800b-c256e3ed545d None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:19.714313 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0364ff1-d76c-4119-800b-c256e3ed545d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:19.724787 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0364ff1-d76c-4119-800b-c256e3ed545d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:50:19.729944 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-d0364ff1-d76c-4119-800b-c256e3ed545d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 14 02:50:19.729944 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:50:19.729944 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:50:19.729944 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:50:19.729944 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:50:19.729944 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:50:19.729944 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:50:19.729944 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:50:19.729944 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:50:19.729944 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:50:19.729944 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:50:19.729944 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:50:19.729944 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:50:19.729944 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:50:19.729944 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:50:19.729944 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:50:19.730935 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:50:19.730935 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:50:19.730935 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:50:19.730935 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:50:19.730935 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:50:19.730935 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:50:19.730935 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:50:19.730935 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:50:19.730935 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:50:19.730935 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:50:19.730935 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:50:19.730935 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:50:19.730935 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:50:19.730935 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:50:19.730935 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:50:19.730935 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:50:19.731766 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:50:19.731766 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:50:19.731766 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:50:19.731766 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:50:19.731766 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:50:19.731766 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:50:19.731766 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:50:19.731766 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:50:19.731766 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:50:19.731766 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:50:19.731766 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 14 02:50:19.731766 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:50:19.734147 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 407/795] 10.222.0.39 () {56 vars in 1120 bytes} [Fri Oct 14 02:50:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:50:19.737919 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d300737-db81-4c01-959f-38894e52f724 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:19.777657 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:19.836578 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d300737-db81-4c01-959f-38894e52f724 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:19.872258 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ef5e45-175f-40cb-8f54-d66e8aa737ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:19.873162 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ef5e45-175f-40cb-8f54-d66e8aa737ae None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:19.873649 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ef5e45-175f-40cb-8f54-d66e8aa737ae None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:19.874183 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ef5e45-175f-40cb-8f54-d66e8aa737ae None admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:19.875046 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ef5e45-175f-40cb-8f54-d66e8aa737ae None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:50:19.875554 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ef5e45-175f-40cb-8f54-d66e8aa737ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:19.877140 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ef5e45-175f-40cb-8f54-d66e8aa737ae None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:19.890079 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 389/796] 10.222.0.39 () {56 vars in 1115 bytes} [Fri Oct 14 02:50:19 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:50:19.891210 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0364ff1-d76c-4119-800b-c256e3ed545d None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:19.951986 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:19.994298 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0364ff1-d76c-4119-800b-c256e3ed545d None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:20.036877 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2a3de4-06a9-4206-bbfc-d5793a0bb496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:20.037589 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a3de4-06a9-4206-bbfc-d5793a0bb496 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:20.037964 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a3de4-06a9-4206-bbfc-d5793a0bb496 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:20.038302 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a3de4-06a9-4206-bbfc-d5793a0bb496 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:20.048561 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2a3de4-06a9-4206-bbfc-d5793a0bb496 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:20.050151 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2a3de4-06a9-4206-bbfc-d5793a0bb496 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:20.178561 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-8b2a3de4-06a9-4206-bbfc-d5793a0bb496 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:50:20.178561 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:50:20.178561 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:50:20.178561 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:50:20.178561 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:50:20.178561 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:50:20.178561 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:50:20.178561 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:50:20.178561 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:50:20.178561 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:50:20.178561 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:50:20.178561 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:50:20.178561 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:50:20.178561 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:50:20.178561 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:50:20.178561 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:50:20.179729 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:50:20.179729 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:50:20.179729 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:50:20.179729 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:50:20.179729 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:50:20.179729 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:50:20.179729 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:50:20.179729 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:50:20.179729 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:50:20.179729 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:50:20.179729 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:50:20.179729 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:50:20.179729 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:50:20.179729 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:50:20.179729 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:50:20.179729 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:50:20.179729 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:50:20.180816 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:50:20.180816 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:50:20.180816 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:50:20.185572 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 408/797] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:50:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 02:50:20.187099 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ef5e45-175f-40cb-8f54-d66e8aa737ae None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:20.285121 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:20.289082 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ef5e45-175f-40cb-8f54-d66e8aa737ae None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:20.323433 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43c5171-0b45-4f6a-861d-a5a30c611c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:20.324038 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c5171-0b45-4f6a-861d-a5a30c611c97 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:20.324323 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c5171-0b45-4f6a-861d-a5a30c611c97 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:20.324596 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c5171-0b45-4f6a-861d-a5a30c611c97 None admin] PATH_INFO: `/v3/domains` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:20.325231 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43c5171-0b45-4f6a-861d-a5a30c611c97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:50:20.325532 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43c5171-0b45-4f6a-861d-a5a30c611c97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:20.326928 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43c5171-0b45-4f6a-861d-a5a30c611c97 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:20.343612 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 390/798] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Oct 14 02:50:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:50:20.344736 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2a3de4-06a9-4206-bbfc-d5793a0bb496 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:20.391807 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6f53a0-903f-4914-b848-c856f4207223 None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:20.392121 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6f53a0-903f-4914-b848-c856f4207223 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:20.392365 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6f53a0-903f-4914-b848-c856f4207223 None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:20.394423 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 391/799] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:50:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 14 02:50:20.399344 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862e2699-fb36-4f8d-8f20-978d47708851 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:20.399635 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862e2699-fb36-4f8d-8f20-978d47708851 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:20.399909 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862e2699-fb36-4f8d-8f20-978d47708851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:20.418264 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-862e2699-fb36-4f8d-8f20-978d47708851 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:50:20.419450 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-862e2699-fb36-4f8d-8f20-978d47708851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:50:20.446069 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2a3de4-06a9-4206-bbfc-d5793a0bb496 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:20.457129 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:20.486836 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e122b79-7813-48b2-9290-549a907e149b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:20.487920 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e122b79-7813-48b2-9290-549a907e149b None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:20.488380 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e122b79-7813-48b2-9290-549a907e149b None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:20.489148 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e122b79-7813-48b2-9290-549a907e149b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:20.492467 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 392/800] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:50:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 14 02:50:20.496302 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e122b79-7813-48b2-9290-549a907e149b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:20.497770 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e122b79-7813-48b2-9290-549a907e149b None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:20.511261 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-5e122b79-7813-48b2-9290-549a907e149b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 14 02:50:20.511261 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:50:20.511261 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:50:20.511261 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:50:20.511261 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:50:20.511261 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:50:20.511261 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:50:20.511261 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:50:20.511261 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:50:20.511261 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:50:20.511261 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:50:20.511261 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:50:20.511261 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 14 02:50:20.511261 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 14 02:50:20.511261 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 14 02:50:20.511261 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 14 02:50:20.512180 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:50:20.512180 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:50:20.512180 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 14 02:50:20.512180 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:50:20.512180 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 14 02:50:20.512180 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:50:20.512180 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:50:20.512180 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:50:20.512180 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:50:20.512180 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:50:20.512180 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:50:20.512180 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:50:20.512180 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:50:20.512180 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:50:20.512180 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:50:20.512180 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:50:20.512180 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:50:20.512180 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:50:20.518646 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:50:20.518646 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:50:20.518646 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 14 02:50:20.518646 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 14 02:50:20.518646 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 14 02:50:20.518646 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 14 02:50:20.518646 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 14 02:50:20.518646 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 14 02:50:20.518646 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 14 02:50:20.518646 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:50:20.518646 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 409/801] 10.222.0.39 () {56 vars in 1093 bytes} [Fri Oct 14 02:50:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 02:50:20.519288 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862e2699-fb36-4f8d-8f20-978d47708851 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:20.608976 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:20.618398 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862e2699-fb36-4f8d-8f20-978d47708851 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:20.652317 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfec593-db99-4030-8c5a-2620ffe59cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:20.652926 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfec593-db99-4030-8c5a-2620ffe59cd4 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:20.653213 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfec593-db99-4030-8c5a-2620ffe59cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:20.653472 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfec593-db99-4030-8c5a-2620ffe59cd4 None admin] PATH_INFO: `/v3/users` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:20.654223 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfec593-db99-4030-8c5a-2620ffe59cd4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:50:20.654529 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfec593-db99-4030-8c5a-2620ffe59cd4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:20.655948 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfec593-db99-4030-8c5a-2620ffe59cd4 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:20.672859 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 393/802] 10.222.0.39 () {56 vars in 1133 bytes} [Fri Oct 14 02:50:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:50:20.678417 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e122b79-7813-48b2-9290-549a907e149b None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:20.759374 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfec593-db99-4030-8c5a-2620ffe59cd4 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:20.784806 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e122b79-7813-48b2-9290-549a907e149b None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:20.821635 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca79334-e01d-4f52-a549-91614b2a5a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:20.822287 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca79334-e01d-4f52-a549-91614b2a5a13 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:20.822647 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca79334-e01d-4f52-a549-91614b2a5a13 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:20.822921 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca79334-e01d-4f52-a549-91614b2a5a13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:20.833549 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca79334-e01d-4f52-a549-91614b2a5a13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:20.835015 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca79334-e01d-4f52-a549-91614b2a5a13 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:20.849581 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-6ca79334-e01d-4f52-a549-91614b2a5a13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:50:20.849581 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:50:20.849581 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:50:20.849581 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:50:20.849581 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:50:20.849581 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:50:20.849581 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:50:20.849581 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:50:20.849581 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:50:20.849581 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:50:20.849581 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:50:20.849581 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:50:20.849581 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:50:20.849581 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:50:20.849581 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:50:20.849581 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:50:20.850538 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:50:20.850538 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:50:20.850538 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:50:20.850538 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:50:20.850538 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:50:20.850538 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:50:20.850538 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:50:20.850538 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:50:20.850538 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:50:20.850538 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:50:20.850538 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:50:20.850538 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:50:20.850538 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:50:20.850538 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:50:20.850538 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:50:20.850538 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:50:20.850538 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:50:20.856874 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:50:20.856874 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:50:20.856874 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:50:20.856874 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 410/803] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:50:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 02:50:20.856874 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca79334-e01d-4f52-a549-91614b2a5a13 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:20.869785 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfec593-db99-4030-8c5a-2620ffe59cd4 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:20.883479 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:20.909097 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deb72e0-5d71-4213-b49b-801bf44f2906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:20.909711 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deb72e0-5d71-4213-b49b-801bf44f2906 None admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:20.910026 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deb72e0-5d71-4213-b49b-801bf44f2906 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:20.910312 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deb72e0-5d71-4213-b49b-801bf44f2906 None admin] PATH_INFO: `/v3/domains` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:20.911002 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7deb72e0-5d71-4213-b49b-801bf44f2906 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:20.912409 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7deb72e0-5d71-4213-b49b-801bf44f2906 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:20.953858 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.sql.core [None req-7deb72e0-5d71-4213-b49b-801bf44f2906 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 14 02:50:20.953858 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 14 02:50:20.953858 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [parameters: {'id': 'ef716420d264485ca04cd34b82c6e667', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 14 02:50:20.953858 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81518) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 14 02:50:20.961046 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-7deb72e0-5d71-4213-b49b-801bf44f2906 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 14 02:50:20.961046 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:50:20.961046 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 14 02:50:20.961046 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 14 02:50:20.961046 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 14 02:50:20.961046 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 14 02:50:20.961046 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 14 02:50:20.961046 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application result = self._query(query) Oct 14 02:50:20.961046 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 14 02:50:20.961046 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application conn.query(q) Oct 14 02:50:20.961046 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 14 02:50:20.961046 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 14 02:50:20.961046 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 14 02:50:20.961046 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application result.read() Oct 14 02:50:20.961872 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 14 02:50:20.961872 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 14 02:50:20.961872 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 14 02:50:20.961872 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 14 02:50:20.961872 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 14 02:50:20.961872 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 14 02:50:20.961872 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 14 02:50:20.961872 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 14 02:50:20.961872 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 14 02:50:20.961872 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:50:20.961872 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 14 02:50:20.961872 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:50:20.961872 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:50:20.961872 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 14 02:50:20.961872 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 14 02:50:20.961872 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 14 02:50:20.961872 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 14 02:50:20.961872 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 14 02:50:20.967876 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application next(self.gen) Oct 14 02:50:20.967876 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 14 02:50:20.967876 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application yield resource Oct 14 02:50:20.967876 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 14 02:50:20.967876 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application next(self.gen) Oct 14 02:50:20.967876 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 14 02:50:20.967876 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application self.session.rollback() Oct 14 02:50:20.967876 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 14 02:50:20.967876 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application self.force_reraise() Oct 14 02:50:20.967876 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 14 02:50:20.967876 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise self.value Oct 14 02:50:20.967876 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 14 02:50:20.967876 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 14 02:50:20.967876 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 14 02:50:20.967876 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application session.commit() Oct 14 02:50:20.967876 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 14 02:50:20.967876 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 14 02:50:20.968768 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 14 02:50:20.968768 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application self._prepare_impl() Oct 14 02:50:20.968768 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 14 02:50:20.968768 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application self.session.flush() Oct 14 02:50:20.968768 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 14 02:50:20.968768 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application self._flush(objects) Oct 14 02:50:20.968768 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 14 02:50:20.968768 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 14 02:50:20.968768 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 14 02:50:20.968768 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application compat.raise_( Oct 14 02:50:20.968768 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 14 02:50:20.968768 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception Oct 14 02:50:20.968768 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 14 02:50:20.968768 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application flush_context.execute() Oct 14 02:50:20.968768 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 14 02:50:20.968768 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rec.execute(self) Oct 14 02:50:20.968768 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 14 02:50:20.969638 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 14 02:50:20.969638 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 14 02:50:20.969638 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 14 02:50:20.969638 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 14 02:50:20.969638 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 14 02:50:20.969638 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 14 02:50:20.969638 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 14 02:50:20.969638 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 14 02:50:20.969638 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 14 02:50:20.969638 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 14 02:50:20.969638 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 14 02:50:20.969638 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 14 02:50:20.969638 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 14 02:50:20.969638 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 14 02:50:20.969638 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 14 02:50:20.970472 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 14 02:50:20.970472 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception Oct 14 02:50:20.970472 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 14 02:50:20.970472 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 14 02:50:20.970472 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 14 02:50:20.970472 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 14 02:50:20.970472 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 14 02:50:20.970472 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application result = self._query(query) Oct 14 02:50:20.970472 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 14 02:50:20.970472 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application conn.query(q) Oct 14 02:50:20.970472 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 14 02:50:20.970472 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 14 02:50:20.970472 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 14 02:50:20.970472 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application result.read() Oct 14 02:50:20.970472 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 14 02:50:20.970472 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 14 02:50:20.970472 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 14 02:50:20.971389 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 14 02:50:20.971389 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 14 02:50:20.971389 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 14 02:50:20.971389 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 14 02:50:20.971389 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 14 02:50:20.971389 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 14 02:50:20.971389 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 14 02:50:20.971389 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [parameters: {'id': 'ef716420d264485ca04cd34b82c6e667', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 14 02:50:20.971389 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 14 02:50:20.971389 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:50:20.971389 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 02:50:20.971389 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:50:20.971389 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:50:20.971389 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 14 02:50:20.971389 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 14 02:50:20.972183 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 14 02:50:20.972183 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 14 02:50:20.972183 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 14 02:50:20.972183 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:50:20.972183 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 02:50:20.972183 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:50:20.972183 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:50:20.972183 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:50:20.972183 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:50:20.972183 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:50:20.972183 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:50:20.972183 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:50:20.972183 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:50:20.972183 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:50:20.972183 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:50:20.972183 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:50:20.972183 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:50:20.973148 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 14 02:50:20.973148 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 14 02:50:20.973148 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:50:20.973148 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:50:20.973148 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 14 02:50:20.973148 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 14 02:50:20.973148 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:50:20.973148 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:50:20.973148 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 14 02:50:20.973148 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 14 02:50:20.973148 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 14 02:50:20.973148 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:50:20.975496 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca79334-e01d-4f52-a549-91614b2a5a13 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:20.975641 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 394/804] 10.222.0.39 () {60 vars in 1137 bytes} [Fri Oct 14 02:50:20 2022] POST /identity/v3/domains => generated 190 bytes in 204 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 14 02:50:20.975641 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deb72e0-5d71-4213-b49b-801bf44f2906 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:20.983027 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:21.014133 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde20b86-b179-48cc-bdeb-a2a11ba38d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:21.014740 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde20b86-b179-48cc-bdeb-a2a11ba38d7f None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:21.015060 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde20b86-b179-48cc-bdeb-a2a11ba38d7f None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:21.015334 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde20b86-b179-48cc-bdeb-a2a11ba38d7f None admin] PATH_INFO: `/v3/domains` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:21.020516 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde20b86-b179-48cc-bdeb-a2a11ba38d7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:50:21.020516 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde20b86-b179-48cc-bdeb-a2a11ba38d7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:21.020516 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde20b86-b179-48cc-bdeb-a2a11ba38d7f None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:21.033540 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 411/805] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Oct 14 02:50:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:50:21.033540 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f945d377-e751-44f1-9aff-dcdab19bfe0c None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:21.033540 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f945d377-e751-44f1-9aff-dcdab19bfe0c None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:21.033540 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f945d377-e751-44f1-9aff-dcdab19bfe0c None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:21.033540 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 412/806] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:50:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:50:21.037015 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f945d377-e751-44f1-9aff-dcdab19bfe0c None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:21.076036 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deb72e0-5d71-4213-b49b-801bf44f2906 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:21.108570 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4eaf0a3-6754-4ff3-bc4b-87b30118c6d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:21.109195 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eaf0a3-6754-4ff3-bc4b-87b30118c6d4 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:21.109476 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eaf0a3-6754-4ff3-bc4b-87b30118c6d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:21.109751 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eaf0a3-6754-4ff3-bc4b-87b30118c6d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:21.118510 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:21.119913 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4eaf0a3-6754-4ff3-bc4b-87b30118c6d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:21.121371 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4eaf0a3-6754-4ff3-bc4b-87b30118c6d4 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:21.136090 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-b4eaf0a3-6754-4ff3-bc4b-87b30118c6d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:50:21.136090 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:50:21.136090 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:50:21.136090 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:50:21.136090 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:50:21.136090 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:50:21.136090 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:50:21.136090 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:50:21.136090 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:50:21.136090 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:50:21.136090 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:50:21.136090 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:50:21.136090 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:50:21.136090 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:50:21.136090 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:50:21.136090 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:50:21.137064 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:50:21.137064 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:50:21.137064 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:50:21.137064 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:50:21.137064 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:50:21.137064 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:50:21.137064 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:50:21.137064 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:50:21.137064 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:50:21.137064 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:50:21.137064 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:50:21.137064 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:50:21.137064 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:50:21.137064 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:50:21.137064 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:50:21.137064 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:50:21.137064 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:50:21.138005 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:50:21.138005 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:50:21.138005 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:50:21.138005 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 395/807] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:50:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:50:21.138234 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f945d377-e751-44f1-9aff-dcdab19bfe0c None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:21.138593 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2922c4d-e309-44b8-bfdb-6fcb441c6947 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:21.138689 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2922c4d-e309-44b8-bfdb-6fcb441c6947 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:21.138963 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2922c4d-e309-44b8-bfdb-6fcb441c6947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:21.156216 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-f2922c4d-e309-44b8-bfdb-6fcb441c6947 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:50:21.158132 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-f2922c4d-e309-44b8-bfdb-6fcb441c6947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:50:21.172585 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cba10d-a6fb-41c2-b8b8-494196120d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:21.173202 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cba10d-a6fb-41c2-b8b8-494196120d1a None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:21.173484 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cba10d-a6fb-41c2-b8b8-494196120d1a None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:21.173776 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cba10d-a6fb-41c2-b8b8-494196120d1a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:21.180718 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cba10d-a6fb-41c2-b8b8-494196120d1a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:21.182184 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cba10d-a6fb-41c2-b8b8-494196120d1a None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:21.192893 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-75cba10d-a6fb-41c2-b8b8-494196120d1a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:50:21.192893 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:50:21.192893 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:50:21.192893 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:50:21.192893 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:50:21.192893 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:50:21.192893 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:50:21.192893 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:50:21.192893 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:50:21.192893 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:50:21.192893 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:50:21.192893 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:50:21.192893 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 14 02:50:21.192893 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 14 02:50:21.192893 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 14 02:50:21.194004 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 14 02:50:21.194004 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:50:21.194004 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:50:21.194004 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:50:21.194004 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:50:21.194004 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:50:21.194004 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:50:21.194004 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:50:21.194004 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:50:21.194004 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:50:21.194004 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:50:21.194004 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:50:21.194004 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:50:21.194004 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:50:21.194004 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:50:21.194004 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:50:21.194004 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:50:21.194902 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 14 02:50:21.194902 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 14 02:50:21.194902 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 14 02:50:21.194902 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 14 02:50:21.194902 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 14 02:50:21.194902 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 14 02:50:21.194902 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:50:21.194902 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:50:21.194902 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 413/808] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Oct 14 02:50:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:50:21.195418 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cba10d-a6fb-41c2-b8b8-494196120d1a None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:21.214828 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:21.223173 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 396/809] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:50:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:50:21.226864 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2922c4d-e309-44b8-bfdb-6fcb441c6947 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:21.299979 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cba10d-a6fb-41c2-b8b8-494196120d1a None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:21.332630 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2922c4d-e309-44b8-bfdb-6fcb441c6947 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:21.334512 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d072a8-5b5b-49d3-8733-50e34381bc30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:21.335116 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d072a8-5b5b-49d3-8733-50e34381bc30 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:21.335396 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d072a8-5b5b-49d3-8733-50e34381bc30 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:21.335674 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d072a8-5b5b-49d3-8733-50e34381bc30 None admin] PATH_INFO: `/v3/domains` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:21.336315 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d072a8-5b5b-49d3-8733-50e34381bc30 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:50:21.336620 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d072a8-5b5b-49d3-8733-50e34381bc30 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:21.338015 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d072a8-5b5b-49d3-8733-50e34381bc30 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:21.357786 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 414/810] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Oct 14 02:50:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:50:21.378237 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6c4600-d231-4f64-824b-4faf6bc9673c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:21.378867 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6c4600-d231-4f64-824b-4faf6bc9673c None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:21.379174 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6c4600-d231-4f64-824b-4faf6bc9673c None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:21.379442 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6c4600-d231-4f64-824b-4faf6bc9673c None admin] PATH_INFO: `/v3/projects` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:21.380145 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6c4600-d231-4f64-824b-4faf6bc9673c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:50:21.380450 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6c4600-d231-4f64-824b-4faf6bc9673c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:21.381932 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6c4600-d231-4f64-824b-4faf6bc9673c None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:21.394942 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 397/811] 10.222.0.39 () {56 vars in 1148 bytes} [Fri Oct 14 02:50:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:50:21.401358 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d072a8-5b5b-49d3-8733-50e34381bc30 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:21.404315 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:21.489127 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6c4600-d231-4f64-824b-4faf6bc9673c None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:21.519101 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d072a8-5b5b-49d3-8733-50e34381bc30 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:21.552041 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:21.559535 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129075c3-48ef-49a6-9104-8128d491e1b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:21.560662 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129075c3-48ef-49a6-9104-8128d491e1b2 None admin] REQUEST_METHOD: `PUT` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:21.560941 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129075c3-48ef-49a6-9104-8128d491e1b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:21.561211 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129075c3-48ef-49a6-9104-8128d491e1b2 None admin] PATH_INFO: `/v3/projects/dac9f8cd285c44e889a621b8b7fe479e/users/a6b07bf0faca47b28b50990d1720853f/roles/22dff4ef49264f709f154e98d331d850` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:21.566049 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129075c3-48ef-49a6-9104-8128d491e1b2 None admin] RBAC: Authorizing `identity:create_grant(project_id=dac9f8cd285c44e889a621b8b7fe479e, user_id=a6b07bf0faca47b28b50990d1720853f, role_id=22dff4ef49264f709f154e98d331d850)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:21.567957 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129075c3-48ef-49a6-9104-8128d491e1b2 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:21.600432 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6c4600-d231-4f64-824b-4faf6bc9673c None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:21.635929 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f892ba2-c816-47e3-9ac8-d6c6c4827d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:21.636565 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f892ba2-c816-47e3-9ac8-d6c6c4827d60 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:21.636843 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f892ba2-c816-47e3-9ac8-d6c6c4827d60 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:21.637116 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f892ba2-c816-47e3-9ac8-d6c6c4827d60 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:21.643889 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 415/812] 10.222.0.39 () {58 vars in 1439 bytes} [Fri Oct 14 02:50:21 2022] PUT /identity/v3/projects/dac9f8cd285c44e889a621b8b7fe479e/users/a6b07bf0faca47b28b50990d1720853f/roles/22dff4ef49264f709f154e98d331d850 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:50:21.648225 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f892ba2-c816-47e3-9ac8-d6c6c4827d60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:50:21.655035 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-7f892ba2-c816-47e3-9ac8-d6c6c4827d60 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:50:21.655035 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:50:21.655035 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:50:21.655035 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:50:21.655035 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:50:21.655035 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:50:21.655035 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:50:21.655035 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:50:21.655035 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:50:21.655035 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:50:21.655035 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:50:21.655035 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:50:21.655035 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:50:21.655035 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:50:21.655035 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:50:21.655035 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:50:21.656059 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:50:21.656059 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:50:21.656059 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:50:21.656059 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:50:21.656059 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:50:21.656059 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:50:21.656059 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:50:21.656059 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:50:21.656059 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:50:21.656059 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:50:21.656059 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:50:21.656059 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:50:21.656059 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:50:21.656059 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:50:21.656059 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:50:21.656059 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:50:21.656889 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:50:21.656889 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:50:21.656889 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:50:21.656889 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:50:21.656889 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:50:21.656889 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:50:21.656889 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:50:21.656889 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:50:21.656889 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:50:21.656889 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:50:21.656889 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:50:21.656889 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:50:21.656889 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 398/813] 10.222.0.39 () {56 vars in 1096 bytes} [Fri Oct 14 02:50:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 02:50:21.663366 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129075c3-48ef-49a6-9104-8128d491e1b2 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:21.735459 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:21.775575 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129075c3-48ef-49a6-9104-8128d491e1b2 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:21.811217 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6353dab-9546-4fa1-920e-29f452fa2c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:21.811753 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6353dab-9546-4fa1-920e-29f452fa2c14 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:21.812034 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6353dab-9546-4fa1-920e-29f452fa2c14 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:21.812309 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6353dab-9546-4fa1-920e-29f452fa2c14 None admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:21.812975 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6353dab-9546-4fa1-920e-29f452fa2c14 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:50:21.813281 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6353dab-9546-4fa1-920e-29f452fa2c14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:21.814721 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6353dab-9546-4fa1-920e-29f452fa2c14 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:21.823980 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 416/814] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Oct 14 02:50:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:50:21.830073 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f892ba2-c816-47e3-9ac8-d6c6c4827d60 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:21.925925 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f892ba2-c816-47e3-9ac8-d6c6c4827d60 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:21.938808 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:21.958891 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2936757c-bb8a-455a-83fa-1660c8acbfb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:21.959567 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2936757c-bb8a-455a-83fa-1660c8acbfb9 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:21.959846 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2936757c-bb8a-455a-83fa-1660c8acbfb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:21.960118 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2936757c-bb8a-455a-83fa-1660c8acbfb9 None admin] PATH_INFO: `/v3/users/606c9dce3b5f46a391abcbe17b909a5d` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:21.973331 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2936757c-bb8a-455a-83fa-1660c8acbfb9 None admin] RBAC: Authorizing `identity:get_user(user_id=606c9dce3b5f46a391abcbe17b909a5d)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:21.974807 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2936757c-bb8a-455a-83fa-1660c8acbfb9 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:21.976410 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 399/815] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Oct 14 02:50:21 2022] GET /identity/v3/users/606c9dce3b5f46a391abcbe17b909a5d => generated 285 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:50:21.982369 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6353dab-9546-4fa1-920e-29f452fa2c14 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:22.083991 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6353dab-9546-4fa1-920e-29f452fa2c14 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:22.106245 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:22.116522 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf697151-0adf-4725-a51c-2d1f3ecc5eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:22.117716 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf697151-0adf-4725-a51c-2d1f3ecc5eed None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:22.117716 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf697151-0adf-4725-a51c-2d1f3ecc5eed None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:22.117716 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf697151-0adf-4725-a51c-2d1f3ecc5eed None admin] PATH_INFO: `/v3/projects/b31f319b28bc4d1597fa6cd1043a8459` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:22.118793 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf697151-0adf-4725-a51c-2d1f3ecc5eed None admin] RBAC: Authorizing `identity:get_project(project_id=b31f319b28bc4d1597fa6cd1043a8459)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:22.120221 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf697151-0adf-4725-a51c-2d1f3ecc5eed None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:22.121870 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 417/816] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Oct 14 02:50:21 2022] GET /identity/v3/projects/b31f319b28bc4d1597fa6cd1043a8459 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:50:22.128379 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2936757c-bb8a-455a-83fa-1660c8acbfb9 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:22.224280 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2936757c-bb8a-455a-83fa-1660c8acbfb9 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:22.257643 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7e6eb2-460b-45bc-996d-9b24de962550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:22.258321 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7e6eb2-460b-45bc-996d-9b24de962550 None admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:22.258599 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7e6eb2-460b-45bc-996d-9b24de962550 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:22.258869 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7e6eb2-460b-45bc-996d-9b24de962550 None admin] PATH_INFO: `/v3/projects/b31f319b28bc4d1597fa6cd1043a8459/users/606c9dce3b5f46a391abcbe17b909a5d/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:22.260730 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7e6eb2-460b-45bc-996d-9b24de962550 None admin] RBAC: Authorizing `identity:create_grant(project_id=b31f319b28bc4d1597fa6cd1043a8459, user_id=606c9dce3b5f46a391abcbe17b909a5d, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:22.262176 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7e6eb2-460b-45bc-996d-9b24de962550 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:22.348912 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 400/817] 10.222.0.39 () {58 vars in 1439 bytes} [Fri Oct 14 02:50:22 2022] PUT /identity/v3/projects/b31f319b28bc4d1597fa6cd1043a8459/users/606c9dce3b5f46a391abcbe17b909a5d/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:50:23.765462 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5f58db-e7a0-469f-94f9-e05349c37b8b None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:23.766040 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5f58db-e7a0-469f-94f9-e05349c37b8b None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:23.766040 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5f58db-e7a0-469f-94f9-e05349c37b8b None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:23.767370 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 418/818] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:50:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:50:23.772588 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff326a-d44f-4dc8-a2f5-42b9e9713962 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:23.772967 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff326a-d44f-4dc8-a2f5-42b9e9713962 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:23.773170 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff326a-d44f-4dc8-a2f5-42b9e9713962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:23.790674 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-e7ff326a-d44f-4dc8-a2f5-42b9e9713962 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:50:23.791834 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-e7ff326a-d44f-4dc8-a2f5-42b9e9713962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:50:23.816923 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:23.855858 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 401/819] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:50:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:50:24.123290 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5f58db-e7a0-469f-94f9-e05349c37b8b None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:24.221772 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5f58db-e7a0-469f-94f9-e05349c37b8b None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:24.254315 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34a85d6-2f52-4bcf-b9a0-fac30f51b9fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:24.254934 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a85d6-2f52-4bcf-b9a0-fac30f51b9fb None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:24.255213 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a85d6-2f52-4bcf-b9a0-fac30f51b9fb None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:24.255484 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a85d6-2f52-4bcf-b9a0-fac30f51b9fb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:24.265172 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34a85d6-2f52-4bcf-b9a0-fac30f51b9fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:50:24.269959 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-b34a85d6-2f52-4bcf-b9a0-fac30f51b9fb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 14 02:50:24.269959 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:50:24.269959 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:50:24.269959 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:50:24.269959 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:50:24.269959 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:50:24.269959 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:50:24.269959 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:50:24.269959 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:50:24.269959 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:50:24.269959 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:50:24.269959 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:50:24.269959 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:50:24.269959 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:50:24.269959 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:50:24.269959 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:50:24.271025 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:50:24.271025 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:50:24.271025 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:50:24.271025 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:50:24.271025 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:50:24.271025 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:50:24.271025 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:50:24.271025 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:50:24.271025 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:50:24.271025 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:50:24.271025 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:50:24.271025 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:50:24.271025 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:50:24.271025 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:50:24.271025 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:50:24.271025 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:50:24.272038 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:50:24.272038 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:50:24.272038 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:50:24.272038 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:50:24.272038 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:50:24.272038 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:50:24.272038 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:50:24.272038 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:50:24.272038 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:50:24.272038 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:50:24.272038 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 14 02:50:24.272038 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:50:24.274051 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 419/820] 10.222.0.39 () {56 vars in 1120 bytes} [Fri Oct 14 02:50:24 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:50:24.276947 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ff326a-d44f-4dc8-a2f5-42b9e9713962 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:24.376886 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ff326a-d44f-4dc8-a2f5-42b9e9713962 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:24.409955 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84784fa4-8a58-4966-beed-6aabf8537a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:24.410557 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84784fa4-8a58-4966-beed-6aabf8537a7f None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:24.410837 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84784fa4-8a58-4966-beed-6aabf8537a7f None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:24.411110 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84784fa4-8a58-4966-beed-6aabf8537a7f None admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:24.411763 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84784fa4-8a58-4966-beed-6aabf8537a7f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:50:24.412066 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84784fa4-8a58-4966-beed-6aabf8537a7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:24.413425 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84784fa4-8a58-4966-beed-6aabf8537a7f None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:24.421118 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 402/821] 10.222.0.39 () {56 vars in 1115 bytes} [Fri Oct 14 02:50:24 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:50:24.427458 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34a85d6-2f52-4bcf-b9a0-fac30f51b9fb None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:24.451782 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a477c32-1ab1-40df-90fe-de355de446e1 None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:24.452155 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a477c32-1ab1-40df-90fe-de355de446e1 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:24.452433 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a477c32-1ab1-40df-90fe-de355de446e1 None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:24.453738 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 403/822] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:50:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 14 02:50:24.458733 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b10579e-b844-47b4-a7d8-e167b678f0a4 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:24.459018 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b10579e-b844-47b4-a7d8-e167b678f0a4 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:24.459286 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b10579e-b844-47b4-a7d8-e167b678f0a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:24.477030 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-6b10579e-b844-47b4-a7d8-e167b678f0a4 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:50:24.478237 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-6b10579e-b844-47b4-a7d8-e167b678f0a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:50:24.479626 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:24.495986 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:24.527110 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34a85d6-2f52-4bcf-b9a0-fac30f51b9fb None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:24.547143 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 404/823] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:50:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 14 02:50:24.561249 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37c753e-8f8a-474d-bb5f-c83be7442d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:24.561902 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37c753e-8f8a-474d-bb5f-c83be7442d9e None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:24.562179 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37c753e-8f8a-474d-bb5f-c83be7442d9e None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:24.562447 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37c753e-8f8a-474d-bb5f-c83be7442d9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:24.571954 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37c753e-8f8a-474d-bb5f-c83be7442d9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:24.573383 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37c753e-8f8a-474d-bb5f-c83be7442d9e None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:24.587666 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-d37c753e-8f8a-474d-bb5f-c83be7442d9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:50:24.587666 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:50:24.587666 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:50:24.587666 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:50:24.587666 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:50:24.587666 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:50:24.587666 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:50:24.587666 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:50:24.587666 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:50:24.587666 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:50:24.587666 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:50:24.587666 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:50:24.587666 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:50:24.587666 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:50:24.587666 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:50:24.587666 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:50:24.588709 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:50:24.588709 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:50:24.588709 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:50:24.588709 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:50:24.588709 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:50:24.588709 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:50:24.588709 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:50:24.588709 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:50:24.588709 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:50:24.588709 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:50:24.588709 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:50:24.588709 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:50:24.588709 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:50:24.588709 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:50:24.588709 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:50:24.588709 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:50:24.588709 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:50:24.589584 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:50:24.589584 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:50:24.589584 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:50:24.589584 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 420/824] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:50:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 02:50:24.594381 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b10579e-b844-47b4-a7d8-e167b678f0a4 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:24.689660 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b10579e-b844-47b4-a7d8-e167b678f0a4 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:24.722245 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106fcb90-496e-4f8c-9211-dc3b9f898314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:24.722837 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106fcb90-496e-4f8c-9211-dc3b9f898314 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:24.723114 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106fcb90-496e-4f8c-9211-dc3b9f898314 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:24.723384 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106fcb90-496e-4f8c-9211-dc3b9f898314 None admin] PATH_INFO: `/v3/domains` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:24.724018 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106fcb90-496e-4f8c-9211-dc3b9f898314 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:50:24.724318 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106fcb90-496e-4f8c-9211-dc3b9f898314 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:24.725679 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106fcb90-496e-4f8c-9211-dc3b9f898314 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:24.736535 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 405/825] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Oct 14 02:50:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:50:24.742279 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37c753e-8f8a-474d-bb5f-c83be7442d9e None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:24.812285 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106fcb90-496e-4f8c-9211-dc3b9f898314 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:24.828023 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:24.841337 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37c753e-8f8a-474d-bb5f-c83be7442d9e None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:24.843632 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:24.874790 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92886f2-f715-4bdb-8ef2-64a0f663e102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:24.875410 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92886f2-f715-4bdb-8ef2-64a0f663e102 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:24.875692 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92886f2-f715-4bdb-8ef2-64a0f663e102 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:24.875962 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92886f2-f715-4bdb-8ef2-64a0f663e102 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:24.883267 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92886f2-f715-4bdb-8ef2-64a0f663e102 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:24.884793 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92886f2-f715-4bdb-8ef2-64a0f663e102 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:24.897731 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-e92886f2-f715-4bdb-8ef2-64a0f663e102 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 14 02:50:24.897731 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:50:24.897731 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:50:24.897731 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:50:24.897731 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:50:24.897731 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:50:24.897731 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:50:24.897731 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:50:24.897731 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:50:24.897731 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:50:24.897731 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:50:24.897731 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:50:24.897731 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 14 02:50:24.897731 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 14 02:50:24.897731 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 14 02:50:24.897731 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 14 02:50:24.898857 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:50:24.898857 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:50:24.898857 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 14 02:50:24.898857 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:50:24.898857 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 14 02:50:24.898857 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:50:24.898857 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:50:24.898857 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:50:24.898857 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:50:24.898857 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:50:24.898857 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:50:24.898857 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:50:24.898857 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:50:24.898857 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:50:24.898857 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:50:24.898857 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:50:24.898857 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:50:24.898857 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:50:24.900050 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:50:24.900050 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:50:24.900050 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 14 02:50:24.900050 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 14 02:50:24.900050 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 14 02:50:24.900050 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 14 02:50:24.900050 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 14 02:50:24.900050 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 14 02:50:24.900050 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 14 02:50:24.900050 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:50:24.900050 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 421/826] 10.222.0.39 () {56 vars in 1093 bytes} [Fri Oct 14 02:50:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 02:50:24.904467 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92886f2-f715-4bdb-8ef2-64a0f663e102 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:24.914236 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106fcb90-496e-4f8c-9211-dc3b9f898314 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:24.947623 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3e788f-5be7-4482-8d5e-20a22caad36d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:24.948189 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3e788f-5be7-4482-8d5e-20a22caad36d None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:24.948477 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3e788f-5be7-4482-8d5e-20a22caad36d None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:24.948759 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3e788f-5be7-4482-8d5e-20a22caad36d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:24.958948 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3e788f-5be7-4482-8d5e-20a22caad36d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:50:24.964151 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-9b3e788f-5be7-4482-8d5e-20a22caad36d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:50:24.964151 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:50:24.964151 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:50:24.964151 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:50:24.964151 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:50:24.964151 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:50:24.964151 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:50:24.964151 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:50:24.964151 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:50:24.964151 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:50:24.964151 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:50:24.964151 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:50:24.964151 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:50:24.964151 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:50:24.964151 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:50:24.964151 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:50:24.965068 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:50:24.965068 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:50:24.965068 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:50:24.965068 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:50:24.965068 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:50:24.965068 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:50:24.965068 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:50:24.965068 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:50:24.965068 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:50:24.965068 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:50:24.965068 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:50:24.965068 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:50:24.965068 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:50:24.965068 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:50:24.965068 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:50:24.965068 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:50:24.965976 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:50:24.965976 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:50:24.965976 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:50:24.965976 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:50:24.965976 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:50:24.965976 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:50:24.965976 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:50:24.965976 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:50:24.965976 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:50:24.965976 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:50:24.965976 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:50:24.965976 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:50:24.965976 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 406/827] 10.222.0.39 () {56 vars in 1096 bytes} [Fri Oct 14 02:50:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 02:50:24.971097 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3e788f-5be7-4482-8d5e-20a22caad36d None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:25.000128 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92886f2-f715-4bdb-8ef2-64a0f663e102 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:25.037751 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d363c2-e810-4642-a698-455d1215002a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:25.038494 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d363c2-e810-4642-a698-455d1215002a None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:25.038783 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d363c2-e810-4642-a698-455d1215002a None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:25.039064 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d363c2-e810-4642-a698-455d1215002a None admin] PATH_INFO: `/v3/users` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:25.039781 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d363c2-e810-4642-a698-455d1215002a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:50:25.040084 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d363c2-e810-4642-a698-455d1215002a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:25.041514 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d363c2-e810-4642-a698-455d1215002a None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:25.043931 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:25.058665 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 422/828] 10.222.0.39 () {56 vars in 1133 bytes} [Fri Oct 14 02:50:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:50:25.064939 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d363c2-e810-4642-a698-455d1215002a None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:25.067743 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3e788f-5be7-4482-8d5e-20a22caad36d None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:25.088439 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:25.101840 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef40ca4-2741-4bf5-be7f-cbe0c4f75349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:25.102442 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef40ca4-2741-4bf5-be7f-cbe0c4f75349 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:25.102723 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef40ca4-2741-4bf5-be7f-cbe0c4f75349 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:25.102999 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef40ca4-2741-4bf5-be7f-cbe0c4f75349 None admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:25.103651 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef40ca4-2741-4bf5-be7f-cbe0c4f75349 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:50:25.103952 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef40ca4-2741-4bf5-be7f-cbe0c4f75349 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:25.105555 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef40ca4-2741-4bf5-be7f-cbe0c4f75349 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:25.113632 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 407/829] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Oct 14 02:50:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:50:25.119579 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef40ca4-2741-4bf5-be7f-cbe0c4f75349 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:25.164969 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d363c2-e810-4642-a698-455d1215002a None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:25.202278 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eec8fd-dbca-4b44-9baa-88bca57440ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:25.202799 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eec8fd-dbca-4b44-9baa-88bca57440ad None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:25.203077 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eec8fd-dbca-4b44-9baa-88bca57440ad None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:25.203348 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eec8fd-dbca-4b44-9baa-88bca57440ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:25.225063 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef40ca4-2741-4bf5-be7f-cbe0c4f75349 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:25.258152 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fe67fe-9dfa-415b-b608-7dd1f294a32a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:25.258916 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fe67fe-9dfa-415b-b608-7dd1f294a32a None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:25.259510 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fe67fe-9dfa-415b-b608-7dd1f294a32a None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:25.259788 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fe67fe-9dfa-415b-b608-7dd1f294a32a None admin] PATH_INFO: `/v3/users/606c9dce3b5f46a391abcbe17b909a5d` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:25.260901 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fe67fe-9dfa-415b-b608-7dd1f294a32a None admin] RBAC: Authorizing `identity:get_user(user_id=606c9dce3b5f46a391abcbe17b909a5d)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:25.262364 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fe67fe-9dfa-415b-b608-7dd1f294a32a None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:25.263937 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 408/830] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Oct 14 02:50:25 2022] GET /identity/v3/users/606c9dce3b5f46a391abcbe17b909a5d => generated 285 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:50:25.269659 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fe67fe-9dfa-415b-b608-7dd1f294a32a None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:25.271365 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92eec8fd-dbca-4b44-9baa-88bca57440ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:25.272815 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92eec8fd-dbca-4b44-9baa-88bca57440ad None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:25.273114 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:25.296130 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:25.365100 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fe67fe-9dfa-415b-b608-7dd1f294a32a None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:25.370263 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 423/831] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Oct 14 02:50:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 14 02:50:25.375942 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eec8fd-dbca-4b44-9baa-88bca57440ad None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:25.397720 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31960bfb-acfe-4be6-8282-310851d68d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:25.398346 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31960bfb-acfe-4be6-8282-310851d68d4c None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:25.398633 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31960bfb-acfe-4be6-8282-310851d68d4c None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:25.398910 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31960bfb-acfe-4be6-8282-310851d68d4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:25.464116 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31960bfb-acfe-4be6-8282-310851d68d4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:25.465478 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31960bfb-acfe-4be6-8282-310851d68d4c None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:25.467975 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:25.472057 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eec8fd-dbca-4b44-9baa-88bca57440ad None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:25.494555 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:25.505640 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7a8e78-1bc0-4358-9a1a-90604f7cfd76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:25.506382 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7a8e78-1bc0-4358-9a1a-90604f7cfd76 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:25.509072 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7a8e78-1bc0-4358-9a1a-90604f7cfd76 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:25.509072 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7a8e78-1bc0-4358-9a1a-90604f7cfd76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:25.517008 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7a8e78-1bc0-4358-9a1a-90604f7cfd76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:25.518564 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7a8e78-1bc0-4358-9a1a-90604f7cfd76 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:25.532040 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-aa7a8e78-1bc0-4358-9a1a-90604f7cfd76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:50:25.532040 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:50:25.532040 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:50:25.532040 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:50:25.532040 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:50:25.532040 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:50:25.532040 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:50:25.532040 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:50:25.532040 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:50:25.532040 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:50:25.532040 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:50:25.532040 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:50:25.532040 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:50:25.532040 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:50:25.532040 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:50:25.532040 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:50:25.533084 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:50:25.533084 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:50:25.533084 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:50:25.533084 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:50:25.533084 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:50:25.533084 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:50:25.533084 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:50:25.533084 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:50:25.533084 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:50:25.533084 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:50:25.533084 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:50:25.533084 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:50:25.533084 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:50:25.533084 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:50:25.533084 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:50:25.533084 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:50:25.533084 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:50:25.534138 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:50:25.534138 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:50:25.534138 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:50:25.534138 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 424/832] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:50:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 02:50:25.538982 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7a8e78-1bc0-4358-9a1a-90604f7cfd76 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:25.644772 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7a8e78-1bc0-4358-9a1a-90604f7cfd76 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:25.676691 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 409/833] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Oct 14 02:50:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 14 02:50:25.678549 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea170a53-fea9-4a70-b37f-b493fa870c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:25.679149 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea170a53-fea9-4a70-b37f-b493fa870c25 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:25.679430 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea170a53-fea9-4a70-b37f-b493fa870c25 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:25.679704 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea170a53-fea9-4a70-b37f-b493fa870c25 None admin] PATH_INFO: `/v3/domains` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:25.680339 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea170a53-fea9-4a70-b37f-b493fa870c25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:50:25.680644 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea170a53-fea9-4a70-b37f-b493fa870c25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:25.682061 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea170a53-fea9-4a70-b37f-b493fa870c25 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:25.682502 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31960bfb-acfe-4be6-8282-310851d68d4c None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:25.694294 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 425/834] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Oct 14 02:50:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:50:25.699572 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea170a53-fea9-4a70-b37f-b493fa870c25 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:25.731840 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:25.780745 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31960bfb-acfe-4be6-8282-310851d68d4c None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:25.783012 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:25.795173 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea170a53-fea9-4a70-b37f-b493fa870c25 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:25.814522 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be55706b-d420-48c5-b06e-134306a4941d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:25.815281 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be55706b-d420-48c5-b06e-134306a4941d None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:25.815755 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be55706b-d420-48c5-b06e-134306a4941d None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:25.816192 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be55706b-d420-48c5-b06e-134306a4941d None admin] PATH_INFO: `/v3/projects/b31f319b28bc4d1597fa6cd1043a8459` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:25.817547 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be55706b-d420-48c5-b06e-134306a4941d None admin] RBAC: Authorizing `identity:get_project(project_id=b31f319b28bc4d1597fa6cd1043a8459)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:25.819011 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be55706b-d420-48c5-b06e-134306a4941d None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:25.820652 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 410/835] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Oct 14 02:50:25 2022] GET /identity/v3/projects/b31f319b28bc4d1597fa6cd1043a8459 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:50:25.826587 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be55706b-d420-48c5-b06e-134306a4941d None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:25.828799 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9bb6ca-adfa-4653-962b-992eebb2016f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:25.829494 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9bb6ca-adfa-4653-962b-992eebb2016f None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:25.829900 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9bb6ca-adfa-4653-962b-992eebb2016f None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:25.830227 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9bb6ca-adfa-4653-962b-992eebb2016f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:25.836830 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9bb6ca-adfa-4653-962b-992eebb2016f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:25.838380 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9bb6ca-adfa-4653-962b-992eebb2016f None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:25.849787 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-2d9bb6ca-adfa-4653-962b-992eebb2016f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:50:25.849787 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:50:25.849787 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:50:25.849787 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:50:25.849787 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:50:25.849787 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:50:25.849787 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:50:25.849787 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:50:25.849787 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:50:25.849787 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:50:25.849787 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:50:25.849787 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:50:25.849787 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 14 02:50:25.849787 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 14 02:50:25.849787 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 14 02:50:25.851241 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 14 02:50:25.851241 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:50:25.851241 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:50:25.851241 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:50:25.851241 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:50:25.851241 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:50:25.851241 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:50:25.851241 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:50:25.851241 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:50:25.851241 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:50:25.851241 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:50:25.851241 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:50:25.851241 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:50:25.851241 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:50:25.851241 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:50:25.851241 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:50:25.851241 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:50:25.852368 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 14 02:50:25.852368 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 14 02:50:25.852368 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 14 02:50:25.852368 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 14 02:50:25.852368 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 14 02:50:25.852368 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 14 02:50:25.852368 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:50:25.852368 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:50:25.852368 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 426/836] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Oct 14 02:50:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 02:50:25.856810 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9bb6ca-adfa-4653-962b-992eebb2016f None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:25.922955 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be55706b-d420-48c5-b06e-134306a4941d None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:25.944879 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:25.951450 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9bb6ca-adfa-4653-962b-992eebb2016f None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:25.957496 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2988f4-7e8c-42d2-b972-ba25d34ae5ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:25.957496 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2988f4-7e8c-42d2-b972-ba25d34ae5ee None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:25.957496 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2988f4-7e8c-42d2-b972-ba25d34ae5ee None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:25.957496 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2988f4-7e8c-42d2-b972-ba25d34ae5ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:25.958082 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2988f4-7e8c-42d2-b972-ba25d34ae5ee None admin] RBAC: Adding query filter params (role.id=9e86d182c0594e0785b72ba5f390a114, scope.project.id=b31f319b28bc4d1597fa6cd1043a8459, user.id=606c9dce3b5f46a391abcbe17b909a5d) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:50:25.958390 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2988f4-7e8c-42d2-b972-ba25d34ae5ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:25.959794 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2988f4-7e8c-42d2-b972-ba25d34ae5ee None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:25.965460 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 411/837] 10.222.0.39 () {56 vars in 1374 bytes} [Fri Oct 14 02:50:25 2022] GET /identity/v3/role_assignments?user.id=606c9dce3b5f46a391abcbe17b909a5d&scope.project.id=b31f319b28bc4d1597fa6cd1043a8459&role.id=9e86d182c0594e0785b72ba5f390a114 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:50:25.987174 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90936e5-901a-4b92-8c1f-43afe6a28fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:25.987714 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90936e5-901a-4b92-8c1f-43afe6a28fa3 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:25.987994 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90936e5-901a-4b92-8c1f-43afe6a28fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:25.988265 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90936e5-901a-4b92-8c1f-43afe6a28fa3 None admin] PATH_INFO: `/v3/projects` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:25.988964 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90936e5-901a-4b92-8c1f-43afe6a28fa3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:50:25.989268 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90936e5-901a-4b92-8c1f-43afe6a28fa3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:25.990765 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90936e5-901a-4b92-8c1f-43afe6a28fa3 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:26.005801 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 427/838] 10.222.0.39 () {56 vars in 1148 bytes} [Fri Oct 14 02:50:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:50:26.010462 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2988f4-7e8c-42d2-b972-ba25d34ae5ee None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:26.047468 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:26.113340 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2988f4-7e8c-42d2-b972-ba25d34ae5ee None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:26.147614 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbe09e2-7588-44c8-a9cc-cd58b6775a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:26.148215 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbe09e2-7588-44c8-a9cc-cd58b6775a29 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:26.148499 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbe09e2-7588-44c8-a9cc-cd58b6775a29 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:26.148764 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbe09e2-7588-44c8-a9cc-cd58b6775a29 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:26.149476 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbe09e2-7588-44c8-a9cc-cd58b6775a29 None admin] RBAC: Adding query filter params (role.id=22dff4ef49264f709f154e98d331d850, scope.project.id=dac9f8cd285c44e889a621b8b7fe479e, user.id=a6b07bf0faca47b28b50990d1720853f) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:50:26.149809 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbe09e2-7588-44c8-a9cc-cd58b6775a29 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:26.151218 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbe09e2-7588-44c8-a9cc-cd58b6775a29 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:26.156897 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 412/839] 10.222.0.39 () {56 vars in 1374 bytes} [Fri Oct 14 02:50:26 2022] GET /identity/v3/role_assignments?user.id=a6b07bf0faca47b28b50990d1720853f&scope.project.id=dac9f8cd285c44e889a621b8b7fe479e&role.id=22dff4ef49264f709f154e98d331d850 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:50:28.112832 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d490ff-a6c6-4cfb-a112-079767f583c3 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:28.113383 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d490ff-a6c6-4cfb-a112-079767f583c3 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:28.113383 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d490ff-a6c6-4cfb-a112-079767f583c3 None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:28.114736 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 428/840] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:50:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:50:28.119926 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfc3eea-9a0e-4212-9659-04f0b22bdb9c None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:28.120337 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfc3eea-9a0e-4212-9659-04f0b22bdb9c None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:28.120472 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfc3eea-9a0e-4212-9659-04f0b22bdb9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:28.138614 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-ebfc3eea-9a0e-4212-9659-04f0b22bdb9c None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:50:28.139797 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-ebfc3eea-9a0e-4212-9659-04f0b22bdb9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:50:28.202795 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 413/841] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:50:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:50:28.466689 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d490ff-a6c6-4cfb-a112-079767f583c3 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:28.563892 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d490ff-a6c6-4cfb-a112-079767f583c3 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:28.597005 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e7f9f3-ae37-4377-a5e8-296b02fc0a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:28.597616 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e7f9f3-ae37-4377-a5e8-296b02fc0a38 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:28.597927 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e7f9f3-ae37-4377-a5e8-296b02fc0a38 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:28.598202 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e7f9f3-ae37-4377-a5e8-296b02fc0a38 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:28.599291 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e7f9f3-ae37-4377-a5e8-296b02fc0a38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:28.600724 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e7f9f3-ae37-4377-a5e8-296b02fc0a38 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:28.602292 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 429/842] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:50:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:50:28.608088 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfc3eea-9a0e-4212-9659-04f0b22bdb9c None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:28.636519 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:28.702726 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfc3eea-9a0e-4212-9659-04f0b22bdb9c None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:28.735763 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6721238-47df-4003-965b-50dd762aa4c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:28.736333 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6721238-47df-4003-965b-50dd762aa4c7 None admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:28.736613 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6721238-47df-4003-965b-50dd762aa4c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:28.736886 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6721238-47df-4003-965b-50dd762aa4c7 None admin] PATH_INFO: `/v3/users` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:28.737540 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6721238-47df-4003-965b-50dd762aa4c7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:28.738989 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6721238-47df-4003-965b-50dd762aa4c7 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:28.821965 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 414/843] 10.222.0.39 () {60 vars in 1132 bytes} [Fri Oct 14 02:50:28 2022] POST /identity/v3/users => generated 286 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:50:30.945862 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7a97c9-c079-4717-b6ac-16a07d97bb08 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:30.946416 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7a97c9-c079-4717-b6ac-16a07d97bb08 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:30.946416 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7a97c9-c079-4717-b6ac-16a07d97bb08 None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:30.947731 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 430/844] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:50:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:50:30.952943 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20e8ba0-a7ed-4496-b6fd-8333ce6bc8f7 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:30.953346 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20e8ba0-a7ed-4496-b6fd-8333ce6bc8f7 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:30.953495 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20e8ba0-a7ed-4496-b6fd-8333ce6bc8f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:30.971589 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-a20e8ba0-a7ed-4496-b6fd-8333ce6bc8f7 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:50:30.972767 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-a20e8ba0-a7ed-4496-b6fd-8333ce6bc8f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:50:31.037080 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 415/845] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:50:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:50:31.310811 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7a97c9-c079-4717-b6ac-16a07d97bb08 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:31.334848 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:31.409653 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7a97c9-c079-4717-b6ac-16a07d97bb08 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:31.441697 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54294326-2aa5-4460-97ae-2d878a38c4a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:31.442340 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54294326-2aa5-4460-97ae-2d878a38c4a9 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:31.442624 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54294326-2aa5-4460-97ae-2d878a38c4a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:31.442899 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54294326-2aa5-4460-97ae-2d878a38c4a9 None admin] PATH_INFO: `/v3/roles/d695dc72739e4aa7a50d5fb73ad1a91f` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:31.444292 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54294326-2aa5-4460-97ae-2d878a38c4a9 None admin] RBAC: Authorizing `identity:get_role(role_id=d695dc72739e4aa7a50d5fb73ad1a91f)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:31.445654 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54294326-2aa5-4460-97ae-2d878a38c4a9 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:31.446983 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 431/846] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Oct 14 02:50:31 2022] GET /identity/v3/roles/d695dc72739e4aa7a50d5fb73ad1a91f => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:50:31.452637 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20e8ba0-a7ed-4496-b6fd-8333ce6bc8f7 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:31.469501 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:31.546749 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20e8ba0-a7ed-4496-b6fd-8333ce6bc8f7 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:31.578351 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4ab3fc-c85c-4fa5-bb10-abfa18fc40af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:31.578970 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4ab3fc-c85c-4fa5-bb10-abfa18fc40af None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:31.579250 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4ab3fc-c85c-4fa5-bb10-abfa18fc40af None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:31.579522 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4ab3fc-c85c-4fa5-bb10-abfa18fc40af None admin] PATH_INFO: `/v3/users/282baca5fbd84048997f132f38051ee9` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:31.592322 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4ab3fc-c85c-4fa5-bb10-abfa18fc40af None admin] RBAC: Authorizing `identity:get_user(user_id=282baca5fbd84048997f132f38051ee9)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:31.593744 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4ab3fc-c85c-4fa5-bb10-abfa18fc40af None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:31.595325 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 416/847] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Oct 14 02:50:31 2022] GET /identity/v3/users/282baca5fbd84048997f132f38051ee9 => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:50:31.601339 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54294326-2aa5-4460-97ae-2d878a38c4a9 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:31.687688 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:31.697424 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54294326-2aa5-4460-97ae-2d878a38c4a9 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:31.731130 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960317ec-a3f7-4288-825a-61f7b5c9af81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:31.731750 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960317ec-a3f7-4288-825a-61f7b5c9af81 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:31.732031 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960317ec-a3f7-4288-825a-61f7b5c9af81 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:31.732305 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960317ec-a3f7-4288-825a-61f7b5c9af81 None admin] PATH_INFO: `/v3/projects/b31f319b28bc4d1597fa6cd1043a8459` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:31.733408 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960317ec-a3f7-4288-825a-61f7b5c9af81 None admin] RBAC: Authorizing `identity:get_project(project_id=b31f319b28bc4d1597fa6cd1043a8459)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:31.734874 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960317ec-a3f7-4288-825a-61f7b5c9af81 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:31.736509 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 432/848] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Oct 14 02:50:31 2022] GET /identity/v3/projects/b31f319b28bc4d1597fa6cd1043a8459 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:50:31.742402 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4ab3fc-c85c-4fa5-bb10-abfa18fc40af None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:31.812768 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:31.837065 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4ab3fc-c85c-4fa5-bb10-abfa18fc40af None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:31.868825 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f686d1d-1f44-40bd-8884-cc5ddd2d51ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:31.869460 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f686d1d-1f44-40bd-8884-cc5ddd2d51ce None admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:31.869734 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f686d1d-1f44-40bd-8884-cc5ddd2d51ce None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:31.870032 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f686d1d-1f44-40bd-8884-cc5ddd2d51ce None admin] PATH_INFO: `/v3/projects/b31f319b28bc4d1597fa6cd1043a8459/users/282baca5fbd84048997f132f38051ee9/roles/d695dc72739e4aa7a50d5fb73ad1a91f` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:31.872097 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f686d1d-1f44-40bd-8884-cc5ddd2d51ce None admin] RBAC: Authorizing `identity:create_grant(project_id=b31f319b28bc4d1597fa6cd1043a8459, user_id=282baca5fbd84048997f132f38051ee9, role_id=d695dc72739e4aa7a50d5fb73ad1a91f)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:31.873500 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f686d1d-1f44-40bd-8884-cc5ddd2d51ce None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:31.935980 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 417/849] 10.222.0.39 () {58 vars in 1439 bytes} [Fri Oct 14 02:50:31 2022] PUT /identity/v3/projects/b31f319b28bc4d1597fa6cd1043a8459/users/282baca5fbd84048997f132f38051ee9/roles/d695dc72739e4aa7a50d5fb73ad1a91f => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:50:34.044749 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa228d46-6420-40b4-b730-d2fd238c9615 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:34.045204 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa228d46-6420-40b4-b730-d2fd238c9615 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:34.045294 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa228d46-6420-40b4-b730-d2fd238c9615 None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:34.046691 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 433/850] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:50:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:50:34.051962 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5184e98d-e00e-40b7-88b4-732882c4b7bb None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:34.052360 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5184e98d-e00e-40b7-88b4-732882c4b7bb None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:34.052494 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5184e98d-e00e-40b7-88b4-732882c4b7bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:34.070555 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-5184e98d-e00e-40b7-88b4-732882c4b7bb None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:50:34.071716 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-5184e98d-e00e-40b7-88b4-732882c4b7bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:50:34.134369 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 418/851] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:50:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:50:34.400480 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa228d46-6420-40b4-b730-d2fd238c9615 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:34.497815 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa228d46-6420-40b4-b730-d2fd238c9615 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:34.530075 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9667c086-d279-4f6c-bbed-2382179adcca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:34.530687 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9667c086-d279-4f6c-bbed-2382179adcca None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:34.530968 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9667c086-d279-4f6c-bbed-2382179adcca None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:34.531240 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9667c086-d279-4f6c-bbed-2382179adcca None admin] PATH_INFO: `/v3/roles/d695dc72739e4aa7a50d5fb73ad1a91f` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:34.532611 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9667c086-d279-4f6c-bbed-2382179adcca None admin] RBAC: Authorizing `identity:get_role(role_id=d695dc72739e4aa7a50d5fb73ad1a91f)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:34.534004 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9667c086-d279-4f6c-bbed-2382179adcca None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:34.535290 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 434/852] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Oct 14 02:50:34 2022] GET /identity/v3/roles/d695dc72739e4aa7a50d5fb73ad1a91f => generated 223 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:50:34.540876 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5184e98d-e00e-40b7-88b4-732882c4b7bb None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:34.599858 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:34.636497 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5184e98d-e00e-40b7-88b4-732882c4b7bb None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:34.669313 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16b3ff2-2b5c-413a-9c3d-16abc778734b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:34.669911 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16b3ff2-2b5c-413a-9c3d-16abc778734b None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:34.670198 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16b3ff2-2b5c-413a-9c3d-16abc778734b None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:34.670477 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16b3ff2-2b5c-413a-9c3d-16abc778734b None admin] PATH_INFO: `/v3/users/282baca5fbd84048997f132f38051ee9` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:34.671594 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16b3ff2-2b5c-413a-9c3d-16abc778734b None admin] RBAC: Authorizing `identity:get_user(user_id=282baca5fbd84048997f132f38051ee9)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:34.673026 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16b3ff2-2b5c-413a-9c3d-16abc778734b None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:34.675194 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 419/853] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Oct 14 02:50:34 2022] GET /identity/v3/users/282baca5fbd84048997f132f38051ee9 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:50:34.680841 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9667c086-d279-4f6c-bbed-2382179adcca None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:34.706840 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:34.776729 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9667c086-d279-4f6c-bbed-2382179adcca None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:34.809668 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aa53c9-79e8-47a2-b08b-0b18c9ab25f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:34.810273 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aa53c9-79e8-47a2-b08b-0b18c9ab25f3 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:34.810550 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aa53c9-79e8-47a2-b08b-0b18c9ab25f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:34.810818 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aa53c9-79e8-47a2-b08b-0b18c9ab25f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:34.874478 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86aa53c9-79e8-47a2-b08b-0b18c9ab25f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:34.875904 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86aa53c9-79e8-47a2-b08b-0b18c9ab25f3 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:34.899938 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:34.968467 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 435/854] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Oct 14 02:50:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 14 02:50:34.974808 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16b3ff2-2b5c-413a-9c3d-16abc778734b None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:35.069400 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16b3ff2-2b5c-413a-9c3d-16abc778734b None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:35.097354 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:35.102075 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace7f119-511c-4f4b-8255-81e152e372b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:35.106305 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace7f119-511c-4f4b-8255-81e152e372b2 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:35.106305 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace7f119-511c-4f4b-8255-81e152e372b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:35.106305 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace7f119-511c-4f4b-8255-81e152e372b2 None admin] PATH_INFO: `/v3/projects/b31f319b28bc4d1597fa6cd1043a8459` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:35.106305 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace7f119-511c-4f4b-8255-81e152e372b2 None admin] RBAC: Authorizing `identity:get_project(project_id=b31f319b28bc4d1597fa6cd1043a8459)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:35.106305 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace7f119-511c-4f4b-8255-81e152e372b2 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:35.107583 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 420/855] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Oct 14 02:50:34 2022] GET /identity/v3/projects/b31f319b28bc4d1597fa6cd1043a8459 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:50:35.113851 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aa53c9-79e8-47a2-b08b-0b18c9ab25f3 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:35.208356 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aa53c9-79e8-47a2-b08b-0b18c9ab25f3 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:35.232706 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:35.240843 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d35147-225d-4bfc-bcb5-e22dbc5b5a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:35.255134 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d35147-225d-4bfc-bcb5-e22dbc5b5a29 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:35.255134 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d35147-225d-4bfc-bcb5-e22dbc5b5a29 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:35.255134 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d35147-225d-4bfc-bcb5-e22dbc5b5a29 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:35.255134 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d35147-225d-4bfc-bcb5-e22dbc5b5a29 None admin] RBAC: Adding query filter params (role.id=d695dc72739e4aa7a50d5fb73ad1a91f, scope.project.id=b31f319b28bc4d1597fa6cd1043a8459, user.id=282baca5fbd84048997f132f38051ee9) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:50:35.255134 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d35147-225d-4bfc-bcb5-e22dbc5b5a29 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:35.255134 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d35147-225d-4bfc-bcb5-e22dbc5b5a29 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:35.255727 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 436/856] 10.222.0.39 () {56 vars in 1374 bytes} [Fri Oct 14 02:50:35 2022] GET /identity/v3/role_assignments?user.id=282baca5fbd84048997f132f38051ee9&scope.project.id=b31f319b28bc4d1597fa6cd1043a8459&role.id=d695dc72739e4aa7a50d5fb73ad1a91f => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:50:37.362303 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29274944-6353-44f3-9b39-e7a531bea56f None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:37.362812 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29274944-6353-44f3-9b39-e7a531bea56f None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:37.362926 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29274944-6353-44f3-9b39-e7a531bea56f None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:37.364159 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 421/857] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:50:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:50:37.369304 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca8aaca-ee5a-494b-b8a8-0f1b52eca919 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:37.369734 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca8aaca-ee5a-494b-b8a8-0f1b52eca919 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:37.369883 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca8aaca-ee5a-494b-b8a8-0f1b52eca919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:37.387756 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-2ca8aaca-ee5a-494b-b8a8-0f1b52eca919 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:50:37.388946 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-2ca8aaca-ee5a-494b-b8a8-0f1b52eca919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:50:37.452696 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 437/858] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:50:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:50:37.718129 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29274944-6353-44f3-9b39-e7a531bea56f None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:37.812513 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29274944-6353-44f3-9b39-e7a531bea56f None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:37.845433 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c574c6-8fee-4f2c-8a92-17dd1c8bc771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:37.846069 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c574c6-8fee-4f2c-8a92-17dd1c8bc771 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:37.846347 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c574c6-8fee-4f2c-8a92-17dd1c8bc771 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:37.846615 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c574c6-8fee-4f2c-8a92-17dd1c8bc771 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:37.847700 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c574c6-8fee-4f2c-8a92-17dd1c8bc771 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:37.849112 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c574c6-8fee-4f2c-8a92-17dd1c8bc771 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:37.850647 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 422/859] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:50:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:50:37.856645 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca8aaca-ee5a-494b-b8a8-0f1b52eca919 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:37.950457 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca8aaca-ee5a-494b-b8a8-0f1b52eca919 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:37.969944 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:37.983395 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c22618a-3ba7-4652-b6de-a97fd83ae3e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:37.992477 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c22618a-3ba7-4652-b6de-a97fd83ae3e1 None admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:37.992477 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c22618a-3ba7-4652-b6de-a97fd83ae3e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:37.992477 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c22618a-3ba7-4652-b6de-a97fd83ae3e1 None admin] PATH_INFO: `/v3/projects` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:37.992477 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c22618a-3ba7-4652-b6de-a97fd83ae3e1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:37.992477 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c22618a-3ba7-4652-b6de-a97fd83ae3e1 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:38.069726 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 438/860] 10.222.0.39 () {60 vars in 1141 bytes} [Fri Oct 14 02:50:37 2022] POST /identity/v3/projects => generated 311 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 02:50:40.211669 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ccc34b-1c9e-4156-97cb-8d0911aa5761 None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:40.212183 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ccc34b-1c9e-4156-97cb-8d0911aa5761 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:40.212300 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ccc34b-1c9e-4156-97cb-8d0911aa5761 None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:40.213529 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 423/861] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:50:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:50:40.218958 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff17f17-24be-47a4-81af-02d4b5884eac None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:40.219365 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff17f17-24be-47a4-81af-02d4b5884eac None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:40.219502 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff17f17-24be-47a4-81af-02d4b5884eac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:40.238253 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-5ff17f17-24be-47a4-81af-02d4b5884eac None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:50:40.239420 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-5ff17f17-24be-47a4-81af-02d4b5884eac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:50:40.302510 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 439/862] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:50:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:50:40.573189 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ccc34b-1c9e-4156-97cb-8d0911aa5761 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:40.638445 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:40.670441 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ccc34b-1c9e-4156-97cb-8d0911aa5761 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:40.702566 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc628525-bc3c-4806-8c01-b94becabdbcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:40.703182 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc628525-bc3c-4806-8c01-b94becabdbcb None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:40.703459 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc628525-bc3c-4806-8c01-b94becabdbcb None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:40.703724 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc628525-bc3c-4806-8c01-b94becabdbcb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:40.705361 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc628525-bc3c-4806-8c01-b94becabdbcb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:40.706800 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc628525-bc3c-4806-8c01-b94becabdbcb None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:40.708311 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 424/863] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:50:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:50:40.713869 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff17f17-24be-47a4-81af-02d4b5884eac None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:40.807248 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff17f17-24be-47a4-81af-02d4b5884eac None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:40.808694 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:40.841734 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9656eaa7-480e-4754-8a0d-cdb99db8b6ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:40.842296 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9656eaa7-480e-4754-8a0d-cdb99db8b6ab None admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:40.842571 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9656eaa7-480e-4754-8a0d-cdb99db8b6ab None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:40.842838 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9656eaa7-480e-4754-8a0d-cdb99db8b6ab None admin] PATH_INFO: `/v3/users` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:40.843482 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9656eaa7-480e-4754-8a0d-cdb99db8b6ab None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:40.844962 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9656eaa7-480e-4754-8a0d-cdb99db8b6ab None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:40.924678 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 440/864] 10.222.0.39 () {60 vars in 1132 bytes} [Fri Oct 14 02:50:40 2022] POST /identity/v3/users => generated 286 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:50:43.053354 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988c425b-f2bb-4e40-8988-67ec4ed61b89 None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:43.053829 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988c425b-f2bb-4e40-8988-67ec4ed61b89 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:43.054014 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988c425b-f2bb-4e40-8988-67ec4ed61b89 None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:43.055348 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 425/865] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:50:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:50:43.060364 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce3360-c5b7-42cd-8f11-2f121973ed3c None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:43.060727 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce3360-c5b7-42cd-8f11-2f121973ed3c None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:43.060914 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce3360-c5b7-42cd-8f11-2f121973ed3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:43.078492 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-24ce3360-c5b7-42cd-8f11-2f121973ed3c None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:50:43.079687 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-24ce3360-c5b7-42cd-8f11-2f121973ed3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:50:43.142689 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 441/866] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:50:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:50:43.410349 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988c425b-f2bb-4e40-8988-67ec4ed61b89 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:43.506328 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988c425b-f2bb-4e40-8988-67ec4ed61b89 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:43.538979 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc8a0b5-0662-4232-854f-781249f04426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:43.539592 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc8a0b5-0662-4232-854f-781249f04426 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:43.539873 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc8a0b5-0662-4232-854f-781249f04426 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:43.540144 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc8a0b5-0662-4232-854f-781249f04426 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:43.542644 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:43.550199 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc8a0b5-0662-4232-854f-781249f04426 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:50:43.555236 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-bfc8a0b5-0662-4232-854f-781249f04426 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:50:43.555236 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:50:43.555236 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:50:43.555236 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:50:43.555236 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:50:43.555236 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:50:43.555236 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:50:43.555236 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:50:43.555236 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:50:43.555236 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:50:43.555236 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:50:43.555236 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:50:43.555236 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:50:43.555236 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:50:43.555236 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:50:43.555236 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:50:43.556396 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:50:43.556396 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:50:43.556396 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:50:43.556396 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:50:43.556396 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:50:43.556396 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:50:43.556396 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:50:43.556396 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:50:43.556396 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:50:43.556396 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:50:43.556396 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:50:43.556396 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:50:43.556396 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:50:43.556396 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:50:43.556396 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:50:43.556396 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:50:43.557330 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:50:43.557330 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:50:43.557330 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:50:43.557330 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:50:43.557330 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:50:43.557330 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:50:43.557330 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:50:43.557330 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:50:43.557330 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:50:43.557330 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:50:43.557330 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:50:43.557330 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:50:43.557330 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 426/867] 10.222.0.39 () {56 vars in 1096 bytes} [Fri Oct 14 02:50:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:50:43.563271 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ce3360-c5b7-42cd-8f11-2f121973ed3c None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:43.646150 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:43.657411 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ce3360-c5b7-42cd-8f11-2f121973ed3c None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:43.689870 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329fba45-7bf4-4f10-955b-bf4fb2101c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:43.690394 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329fba45-7bf4-4f10-955b-bf4fb2101c9f None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:43.690669 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329fba45-7bf4-4f10-955b-bf4fb2101c9f None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:43.690934 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329fba45-7bf4-4f10-955b-bf4fb2101c9f None admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:43.691579 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329fba45-7bf4-4f10-955b-bf4fb2101c9f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:50:43.691877 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329fba45-7bf4-4f10-955b-bf4fb2101c9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:43.693237 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329fba45-7bf4-4f10-955b-bf4fb2101c9f None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:43.700730 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 442/868] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Oct 14 02:50:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:50:43.707043 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc8a0b5-0662-4232-854f-781249f04426 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:43.800561 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc8a0b5-0662-4232-854f-781249f04426 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:43.833021 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f588fd9d-c3c3-43e5-9213-c7a18c73a9a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:43.833631 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f588fd9d-c3c3-43e5-9213-c7a18c73a9a7 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:43.833948 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f588fd9d-c3c3-43e5-9213-c7a18c73a9a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:43.834214 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f588fd9d-c3c3-43e5-9213-c7a18c73a9a7 None admin] PATH_INFO: `/v3/users/06f0db35cfae40faaef63fc126535710` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:43.846537 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f588fd9d-c3c3-43e5-9213-c7a18c73a9a7 None admin] RBAC: Authorizing `identity:get_user(user_id=06f0db35cfae40faaef63fc126535710)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:43.847958 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f588fd9d-c3c3-43e5-9213-c7a18c73a9a7 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:43.849497 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 427/869] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Oct 14 02:50:43 2022] GET /identity/v3/users/06f0db35cfae40faaef63fc126535710 => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:50:43.855194 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329fba45-7bf4-4f10-955b-bf4fb2101c9f None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:43.949967 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329fba45-7bf4-4f10-955b-bf4fb2101c9f None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:43.981873 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aae6e1-7b61-4bc6-b3d9-07815b6641fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:43.982478 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aae6e1-7b61-4bc6-b3d9-07815b6641fb None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:43.982756 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aae6e1-7b61-4bc6-b3d9-07815b6641fb None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:43.983026 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aae6e1-7b61-4bc6-b3d9-07815b6641fb None admin] PATH_INFO: `/v3/projects/165e3cda52434c2091b2948434e34029` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:43.984107 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aae6e1-7b61-4bc6-b3d9-07815b6641fb None admin] RBAC: Authorizing `identity:get_project(project_id=165e3cda52434c2091b2948434e34029)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:43.986066 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aae6e1-7b61-4bc6-b3d9-07815b6641fb None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:43.987656 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 443/870] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Oct 14 02:50:43 2022] GET /identity/v3/projects/165e3cda52434c2091b2948434e34029 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:50:43.993568 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f588fd9d-c3c3-43e5-9213-c7a18c73a9a7 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:44.048709 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:44.089828 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f588fd9d-c3c3-43e5-9213-c7a18c73a9a7 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:44.121686 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83252012-5603-4b4e-b8e1-a1c4661000a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:44.122353 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83252012-5603-4b4e-b8e1-a1c4661000a0 None admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:44.122630 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83252012-5603-4b4e-b8e1-a1c4661000a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:44.122901 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83252012-5603-4b4e-b8e1-a1c4661000a0 None admin] PATH_INFO: `/v3/projects/165e3cda52434c2091b2948434e34029/users/06f0db35cfae40faaef63fc126535710/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:44.124735 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83252012-5603-4b4e-b8e1-a1c4661000a0 None admin] RBAC: Authorizing `identity:create_grant(project_id=165e3cda52434c2091b2948434e34029, user_id=06f0db35cfae40faaef63fc126535710, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:44.126772 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83252012-5603-4b4e-b8e1-a1c4661000a0 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:44.226812 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 428/871] 10.222.0.39 () {58 vars in 1439 bytes} [Fri Oct 14 02:50:43 2022] PUT /identity/v3/projects/165e3cda52434c2091b2948434e34029/users/06f0db35cfae40faaef63fc126535710/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:50:46.321496 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771d097c-f53d-43a9-8a25-38a6296be6ca None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:46.322022 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771d097c-f53d-43a9-8a25-38a6296be6ca None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:46.322111 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771d097c-f53d-43a9-8a25-38a6296be6ca None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:46.323429 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 444/872] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:50:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:50:46.328791 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b7c7b-6796-4207-b144-c2134ed0cee3 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:46.329732 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b7c7b-6796-4207-b144-c2134ed0cee3 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:46.329861 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b7c7b-6796-4207-b144-c2134ed0cee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:46.347933 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-156b7c7b-6796-4207-b144-c2134ed0cee3 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:50:46.349104 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-156b7c7b-6796-4207-b144-c2134ed0cee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:50:46.411752 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 429/873] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:50:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:50:46.674974 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771d097c-f53d-43a9-8a25-38a6296be6ca None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:46.694104 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:46.773404 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771d097c-f53d-43a9-8a25-38a6296be6ca None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:46.805461 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1def40ab-c345-450f-bb38-de70d7cb7582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:46.806100 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1def40ab-c345-450f-bb38-de70d7cb7582 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:46.806382 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1def40ab-c345-450f-bb38-de70d7cb7582 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:46.806652 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1def40ab-c345-450f-bb38-de70d7cb7582 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:46.816915 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1def40ab-c345-450f-bb38-de70d7cb7582 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:50:46.821611 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-1def40ab-c345-450f-bb38-de70d7cb7582 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:50:46.821611 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:50:46.821611 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:50:46.821611 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:50:46.821611 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:50:46.821611 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:50:46.821611 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:50:46.821611 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:50:46.821611 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:50:46.821611 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:50:46.821611 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:50:46.821611 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:50:46.821611 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:50:46.821611 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:50:46.821611 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:50:46.821611 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:50:46.822703 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:50:46.822703 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:50:46.822703 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:50:46.822703 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:50:46.822703 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:50:46.822703 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:50:46.822703 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:50:46.822703 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:50:46.822703 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:50:46.822703 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:50:46.822703 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:50:46.822703 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:50:46.822703 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:50:46.822703 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:50:46.822703 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:50:46.822703 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:50:46.823912 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:50:46.823912 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:50:46.823912 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:50:46.823912 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:50:46.823912 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:50:46.823912 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:50:46.823912 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:50:46.823912 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:50:46.823912 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:50:46.823912 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:50:46.823912 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:50:46.823912 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:50:46.823912 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 445/874] 10.222.0.39 () {56 vars in 1096 bytes} [Fri Oct 14 02:50:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:50:46.828531 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156b7c7b-6796-4207-b144-c2134ed0cee3 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:46.871553 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:46.922108 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156b7c7b-6796-4207-b144-c2134ed0cee3 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:46.954925 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4839705b-f25b-4dd8-a51f-b7f5a6118cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:46.955499 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4839705b-f25b-4dd8-a51f-b7f5a6118cd7 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:46.955777 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4839705b-f25b-4dd8-a51f-b7f5a6118cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:46.956044 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4839705b-f25b-4dd8-a51f-b7f5a6118cd7 None admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:46.956703 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4839705b-f25b-4dd8-a51f-b7f5a6118cd7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:50:46.957001 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4839705b-f25b-4dd8-a51f-b7f5a6118cd7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:46.958382 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4839705b-f25b-4dd8-a51f-b7f5a6118cd7 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:46.966259 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 430/875] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Oct 14 02:50:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:50:46.972138 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1def40ab-c345-450f-bb38-de70d7cb7582 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:47.049917 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:47.071696 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1def40ab-c345-450f-bb38-de70d7cb7582 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:47.104508 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca441882-5c88-47a5-85cd-2260ad9b1351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:47.105122 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca441882-5c88-47a5-85cd-2260ad9b1351 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:47.105404 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca441882-5c88-47a5-85cd-2260ad9b1351 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:47.105680 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca441882-5c88-47a5-85cd-2260ad9b1351 None admin] PATH_INFO: `/v3/users/06f0db35cfae40faaef63fc126535710` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:47.106827 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca441882-5c88-47a5-85cd-2260ad9b1351 None admin] RBAC: Authorizing `identity:get_user(user_id=06f0db35cfae40faaef63fc126535710)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:47.108256 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca441882-5c88-47a5-85cd-2260ad9b1351 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:47.109842 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 446/876] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Oct 14 02:50:46 2022] GET /identity/v3/users/06f0db35cfae40faaef63fc126535710 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:50:47.116128 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4839705b-f25b-4dd8-a51f-b7f5a6118cd7 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:47.209516 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4839705b-f25b-4dd8-a51f-b7f5a6118cd7 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:47.215985 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:47.241383 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a223e8-b91c-45e3-9a6c-e2c1d2cc7088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:47.241986 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a223e8-b91c-45e3-9a6c-e2c1d2cc7088 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:47.242265 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a223e8-b91c-45e3-9a6c-e2c1d2cc7088 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:47.242532 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a223e8-b91c-45e3-9a6c-e2c1d2cc7088 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:47.306225 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a223e8-b91c-45e3-9a6c-e2c1d2cc7088 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:47.307646 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a223e8-b91c-45e3-9a6c-e2c1d2cc7088 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:47.400211 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 431/877] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Oct 14 02:50:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 14 02:50:47.406027 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca441882-5c88-47a5-85cd-2260ad9b1351 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:47.502668 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca441882-5c88-47a5-85cd-2260ad9b1351 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:47.535226 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4b0e33-e536-454c-9f8b-d9dd11c62c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:47.535821 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4b0e33-e536-454c-9f8b-d9dd11c62c01 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:47.536103 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4b0e33-e536-454c-9f8b-d9dd11c62c01 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:47.536378 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4b0e33-e536-454c-9f8b-d9dd11c62c01 None admin] PATH_INFO: `/v3/projects/165e3cda52434c2091b2948434e34029` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:47.537480 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4b0e33-e536-454c-9f8b-d9dd11c62c01 None admin] RBAC: Authorizing `identity:get_project(project_id=165e3cda52434c2091b2948434e34029)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:47.538935 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4b0e33-e536-454c-9f8b-d9dd11c62c01 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:47.540521 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 447/878] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Oct 14 02:50:47 2022] GET /identity/v3/projects/165e3cda52434c2091b2948434e34029 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:50:47.546413 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a223e8-b91c-45e3-9a6c-e2c1d2cc7088 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:47.554704 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:47.641801 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a223e8-b91c-45e3-9a6c-e2c1d2cc7088 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:47.675001 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aceb7c-5985-45ae-8f94-7580215f3e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:47.675581 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aceb7c-5985-45ae-8f94-7580215f3e26 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:47.675856 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aceb7c-5985-45ae-8f94-7580215f3e26 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:47.676128 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aceb7c-5985-45ae-8f94-7580215f3e26 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:47.676837 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aceb7c-5985-45ae-8f94-7580215f3e26 None admin] RBAC: Adding query filter params (role.id=9e86d182c0594e0785b72ba5f390a114, scope.project.id=165e3cda52434c2091b2948434e34029, user.id=06f0db35cfae40faaef63fc126535710) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:50:47.677141 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aceb7c-5985-45ae-8f94-7580215f3e26 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:47.678571 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aceb7c-5985-45ae-8f94-7580215f3e26 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:47.684024 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 432/879] 10.222.0.39 () {56 vars in 1374 bytes} [Fri Oct 14 02:50:47 2022] GET /identity/v3/role_assignments?user.id=06f0db35cfae40faaef63fc126535710&scope.project.id=165e3cda52434c2091b2948434e34029&role.id=9e86d182c0594e0785b72ba5f390a114 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:50:49.779244 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b191d71-81a5-486b-a26d-09ce8a0390b0 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:49.779802 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b191d71-81a5-486b-a26d-09ce8a0390b0 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:49.779802 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b191d71-81a5-486b-a26d-09ce8a0390b0 None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:49.781116 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 448/880] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:50:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:50:49.786497 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6e26ec-d6c3-496d-9ba4-29128042c3d9 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:49.786915 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6e26ec-d6c3-496d-9ba4-29128042c3d9 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:49.787022 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6e26ec-d6c3-496d-9ba4-29128042c3d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:49.804575 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-7e6e26ec-d6c3-496d-9ba4-29128042c3d9 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:50:49.805736 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-7e6e26ec-d6c3-496d-9ba4-29128042c3d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:50:49.864888 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:49.869211 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 433/881] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:50:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:50:50.136735 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b191d71-81a5-486b-a26d-09ce8a0390b0 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:50.152007 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:50.234330 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b191d71-81a5-486b-a26d-09ce8a0390b0 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:50.267108 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c86fd9-aeb8-4c35-b2a5-8b16fd4dc62d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:50.267692 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c86fd9-aeb8-4c35-b2a5-8b16fd4dc62d None admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:50.267968 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c86fd9-aeb8-4c35-b2a5-8b16fd4dc62d None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:50.268239 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c86fd9-aeb8-4c35-b2a5-8b16fd4dc62d None admin] PATH_INFO: `/v3/domains` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:50.268883 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c86fd9-aeb8-4c35-b2a5-8b16fd4dc62d None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:50.270267 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c86fd9-aeb8-4c35-b2a5-8b16fd4dc62d None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:50.320254 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 449/882] 10.222.0.39 () {60 vars in 1138 bytes} [Fri Oct 14 02:50:50 2022] POST /identity/v3/domains => generated 267 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:50:52.474125 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a34e29-548a-4aa4-8070-50230c3be0fd None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:52.474705 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a34e29-548a-4aa4-8070-50230c3be0fd None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:52.474705 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a34e29-548a-4aa4-8070-50230c3be0fd None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:52.475982 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 434/883] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:50:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:50:52.481489 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982b207a-cf66-403c-9e74-74f7860d3d35 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:52.481929 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982b207a-cf66-403c-9e74-74f7860d3d35 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:52.482064 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982b207a-cf66-403c-9e74-74f7860d3d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:52.499614 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-982b207a-cf66-403c-9e74-74f7860d3d35 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:50:52.500789 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-982b207a-cf66-403c-9e74-74f7860d3d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:50:52.564050 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 450/884] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:50:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:50:52.829961 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a34e29-548a-4aa4-8070-50230c3be0fd None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:52.925298 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a34e29-548a-4aa4-8070-50230c3be0fd None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:52.957571 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7efb58-8bfa-4612-b4e4-1b34e93b9b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:52.958213 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7efb58-8bfa-4612-b4e4-1b34e93b9b85 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:52.958494 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7efb58-8bfa-4612-b4e4-1b34e93b9b85 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:52.958766 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7efb58-8bfa-4612-b4e4-1b34e93b9b85 None admin] PATH_INFO: `/v3/domains/20f577128e53440ea04e5ff5d0e42ab0` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:52.968697 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7efb58-8bfa-4612-b4e4-1b34e93b9b85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=20f577128e53440ea04e5ff5d0e42ab0)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:52.970145 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7efb58-8bfa-4612-b4e4-1b34e93b9b85 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:52.971679 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 435/885] 10.222.0.39 () {56 vars in 1183 bytes} [Fri Oct 14 02:50:52 2022] GET /identity/v3/domains/20f577128e53440ea04e5ff5d0e42ab0 => generated 267 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:50:52.977428 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982b207a-cf66-403c-9e74-74f7860d3d35 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:52.980973 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:53.073818 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982b207a-cf66-403c-9e74-74f7860d3d35 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:53.105567 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f148dc54-42a3-493d-9371-90f20333cfd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:53.106125 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148dc54-42a3-493d-9371-90f20333cfd6 None admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:53.106404 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148dc54-42a3-493d-9371-90f20333cfd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:53.106673 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148dc54-42a3-493d-9371-90f20333cfd6 None admin] PATH_INFO: `/v3/projects` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:53.107322 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f148dc54-42a3-493d-9371-90f20333cfd6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:53.108743 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f148dc54-42a3-493d-9371-90f20333cfd6 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:53.182642 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 451/886] 10.222.0.39 () {60 vars in 1141 bytes} [Fri Oct 14 02:50:52 2022] POST /identity/v3/projects => generated 361 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 02:50:55.332339 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4910002b-51bb-4c33-9a67-7b920f0a78b2 None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:55.332847 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4910002b-51bb-4c33-9a67-7b920f0a78b2 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:55.332939 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4910002b-51bb-4c33-9a67-7b920f0a78b2 None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:55.334246 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 436/887] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:50:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:50:55.339498 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d2a0e-6684-4d04-978f-51458ef71062 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:55.339883 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d2a0e-6684-4d04-978f-51458ef71062 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:55.340046 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d2a0e-6684-4d04-978f-51458ef71062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:55.357881 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-5b5d2a0e-6684-4d04-978f-51458ef71062 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:50:55.359052 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-5b5d2a0e-6684-4d04-978f-51458ef71062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:50:55.407356 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:55.423263 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 452/888] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:50:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:50:55.693023 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4910002b-51bb-4c33-9a67-7b920f0a78b2 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:55.751141 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:55.790845 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4910002b-51bb-4c33-9a67-7b920f0a78b2 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:55.822546 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8532b08-ad66-4923-9ef7-d1b358d3ef7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:55.823161 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8532b08-ad66-4923-9ef7-d1b358d3ef7f None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:55.823444 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8532b08-ad66-4923-9ef7-d1b358d3ef7f None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:55.823718 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8532b08-ad66-4923-9ef7-d1b358d3ef7f None admin] PATH_INFO: `/v3/domains/20f577128e53440ea04e5ff5d0e42ab0` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:55.824818 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8532b08-ad66-4923-9ef7-d1b358d3ef7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=20f577128e53440ea04e5ff5d0e42ab0)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:55.826280 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8532b08-ad66-4923-9ef7-d1b358d3ef7f None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:55.827809 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 437/889] 10.222.0.39 () {56 vars in 1183 bytes} [Fri Oct 14 02:50:55 2022] GET /identity/v3/domains/20f577128e53440ea04e5ff5d0e42ab0 => generated 267 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:50:55.833541 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5d2a0e-6684-4d04-978f-51458ef71062 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:55.929468 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5d2a0e-6684-4d04-978f-51458ef71062 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:55.961596 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7491528-66af-4f72-876a-802fc2a8602b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:55.962165 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7491528-66af-4f72-876a-802fc2a8602b None admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:55.962445 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7491528-66af-4f72-876a-802fc2a8602b None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:55.962715 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7491528-66af-4f72-876a-802fc2a8602b None admin] PATH_INFO: `/v3/users` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:55.963370 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7491528-66af-4f72-876a-802fc2a8602b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:55.964791 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7491528-66af-4f72-876a-802fc2a8602b None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:56.079515 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 453/890] 10.222.0.39 () {60 vars in 1132 bytes} [Fri Oct 14 02:50:55 2022] POST /identity/v3/users => generated 311 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 02:50:58.233349 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59ab155-7f56-4245-948f-47bcb7725d2f None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:58.233919 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59ab155-7f56-4245-948f-47bcb7725d2f None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:58.233919 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59ab155-7f56-4245-948f-47bcb7725d2f None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:58.235754 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 438/891] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:50:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:50:58.240983 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8a2921-694b-4e1b-add4-ddac6d90439d None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:58.241398 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8a2921-694b-4e1b-add4-ddac6d90439d None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:58.241574 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8a2921-694b-4e1b-add4-ddac6d90439d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:58.260112 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-8e8a2921-694b-4e1b-add4-ddac6d90439d None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:50:58.261223 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-8e8a2921-694b-4e1b-add4-ddac6d90439d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:50:58.294466 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:58.325725 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 454/892] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:50:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:50:58.595308 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59ab155-7f56-4245-948f-47bcb7725d2f None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:58.692357 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59ab155-7f56-4245-948f-47bcb7725d2f None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:58.716211 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:58.723937 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb324c6-375d-4adb-b254-0108a7f8516c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:58.724546 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb324c6-375d-4adb-b254-0108a7f8516c None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:58.724823 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb324c6-375d-4adb-b254-0108a7f8516c None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:58.725094 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb324c6-375d-4adb-b254-0108a7f8516c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:58.735981 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb324c6-375d-4adb-b254-0108a7f8516c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:50:58.740858 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-4cb324c6-375d-4adb-b254-0108a7f8516c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:50:58.740858 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:50:58.740858 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:50:58.740858 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:50:58.740858 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:50:58.740858 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:50:58.740858 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:50:58.740858 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:50:58.740858 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:50:58.740858 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:50:58.740858 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:50:58.740858 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:50:58.740858 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:50:58.740858 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:50:58.740858 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:50:58.740858 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:50:58.742058 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:50:58.742058 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:50:58.742058 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:50:58.742058 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:50:58.742058 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:50:58.742058 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:50:58.742058 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:50:58.742058 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:50:58.742058 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:50:58.742058 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:50:58.742058 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:50:58.742058 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:50:58.742058 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:50:58.742058 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:50:58.742058 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:50:58.742058 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:50:58.743148 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:50:58.743148 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:50:58.743148 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:50:58.743148 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:50:58.743148 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:50:58.743148 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:50:58.743148 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:50:58.743148 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:50:58.743148 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:50:58.743148 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:50:58.743148 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:50:58.743148 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:50:58.745918 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 439/893] 10.222.0.39 () {56 vars in 1096 bytes} [Fri Oct 14 02:50:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:50:58.747896 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8a2921-694b-4e1b-add4-ddac6d90439d None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:58.842698 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8a2921-694b-4e1b-add4-ddac6d90439d None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:58.876080 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6972fb-a9ef-4460-ae75-668c909758d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:58.876613 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6972fb-a9ef-4460-ae75-668c909758d9 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:58.876892 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6972fb-a9ef-4460-ae75-668c909758d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:58.877166 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6972fb-a9ef-4460-ae75-668c909758d9 None admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:58.877860 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6972fb-a9ef-4460-ae75-668c909758d9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:50:58.878167 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6972fb-a9ef-4460-ae75-668c909758d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:58.879535 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6972fb-a9ef-4460-ae75-668c909758d9 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:58.887240 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 455/894] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Oct 14 02:50:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:50:58.892856 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb324c6-375d-4adb-b254-0108a7f8516c None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:58.988543 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb324c6-375d-4adb-b254-0108a7f8516c None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:59.022767 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5a12fe-1f49-4c18-8b98-9f5c83211a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:59.023389 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a12fe-1f49-4c18-8b98-9f5c83211a07 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:59.023672 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a12fe-1f49-4c18-8b98-9f5c83211a07 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:59.023946 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a12fe-1f49-4c18-8b98-9f5c83211a07 None admin] PATH_INFO: `/v3/users/7f8b15dd7b7241099478fd81f2fe76ad` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:59.037907 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5a12fe-1f49-4c18-8b98-9f5c83211a07 None admin] RBAC: Authorizing `identity:get_user(user_id=7f8b15dd7b7241099478fd81f2fe76ad)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:59.039334 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5a12fe-1f49-4c18-8b98-9f5c83211a07 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:59.040879 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 440/895] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Oct 14 02:50:58 2022] GET /identity/v3/users/7f8b15dd7b7241099478fd81f2fe76ad => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:50:59.046633 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6972fb-a9ef-4460-ae75-668c909758d9 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:59.082260 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:59.146601 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6972fb-a9ef-4460-ae75-668c909758d9 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:59.179480 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51c27cb-6be0-4a07-82e5-d0260085f76f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:59.180095 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51c27cb-6be0-4a07-82e5-d0260085f76f None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:59.180379 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51c27cb-6be0-4a07-82e5-d0260085f76f None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:59.180656 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51c27cb-6be0-4a07-82e5-d0260085f76f None admin] PATH_INFO: `/v3/projects/987e4005e25e4aa8a8e5ff7aca88ea1c` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:59.181791 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51c27cb-6be0-4a07-82e5-d0260085f76f None admin] RBAC: Authorizing `identity:get_project(project_id=987e4005e25e4aa8a8e5ff7aca88ea1c)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:59.183226 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51c27cb-6be0-4a07-82e5-d0260085f76f None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:59.184845 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 456/896] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Oct 14 02:50:59 2022] GET /identity/v3/projects/987e4005e25e4aa8a8e5ff7aca88ea1c => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:50:59.190779 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5a12fe-1f49-4c18-8b98-9f5c83211a07 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:50:59.236108 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:50:59.287145 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5a12fe-1f49-4c18-8b98-9f5c83211a07 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:50:59.318783 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386ca7f0-4ccd-46bc-8991-83814efeb8fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:50:59.319430 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ca7f0-4ccd-46bc-8991-83814efeb8fd None admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:50:59.319711 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ca7f0-4ccd-46bc-8991-83814efeb8fd None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:50:59.320019 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ca7f0-4ccd-46bc-8991-83814efeb8fd None admin] PATH_INFO: `/v3/projects/987e4005e25e4aa8a8e5ff7aca88ea1c/users/7f8b15dd7b7241099478fd81f2fe76ad/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:50:59.321911 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386ca7f0-4ccd-46bc-8991-83814efeb8fd None admin] RBAC: Authorizing `identity:create_grant(project_id=987e4005e25e4aa8a8e5ff7aca88ea1c, user_id=7f8b15dd7b7241099478fd81f2fe76ad, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:50:59.323333 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386ca7f0-4ccd-46bc-8991-83814efeb8fd None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:50:59.386675 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 441/897] 10.222.0.39 () {58 vars in 1439 bytes} [Fri Oct 14 02:50:59 2022] PUT /identity/v3/projects/987e4005e25e4aa8a8e5ff7aca88ea1c/users/7f8b15dd7b7241099478fd81f2fe76ad/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 02:51:01.489383 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27a4992-9535-43ce-88aa-879d968b2a0b None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:51:01.489958 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27a4992-9535-43ce-88aa-879d968b2a0b None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:51:01.490097 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27a4992-9535-43ce-88aa-879d968b2a0b None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:51:01.491451 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 457/898] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:51:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:51:01.496698 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e5ed70-dea1-4500-966e-3a7ef662ef31 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:51:01.497136 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e5ed70-dea1-4500-966e-3a7ef662ef31 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:51:01.497248 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e5ed70-dea1-4500-966e-3a7ef662ef31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:51:01.514799 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-69e5ed70-dea1-4500-966e-3a7ef662ef31 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:51:01.516510 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-69e5ed70-dea1-4500-966e-3a7ef662ef31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:51:01.579037 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 442/899] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:51:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:51:01.844711 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27a4992-9535-43ce-88aa-879d968b2a0b None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:51:01.941968 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27a4992-9535-43ce-88aa-879d968b2a0b None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:51:01.943418 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:51:01.974593 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d08c50a-6da5-4f87-b655-58c58336996f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:51:01.975199 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d08c50a-6da5-4f87-b655-58c58336996f None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:51:01.975479 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d08c50a-6da5-4f87-b655-58c58336996f None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:51:01.975751 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d08c50a-6da5-4f87-b655-58c58336996f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:51:01.986160 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d08c50a-6da5-4f87-b655-58c58336996f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:51:01.991762 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-3d08c50a-6da5-4f87-b655-58c58336996f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:51:01.991762 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:51:01.991762 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:51:01.991762 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:51:01.991762 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:51:01.991762 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:51:01.991762 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:51:01.991762 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:51:01.991762 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:51:01.991762 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:51:01.991762 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:51:01.991762 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:51:01.991762 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:51:01.991762 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:51:01.991762 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:51:01.991762 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:51:01.992844 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:51:01.992844 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:51:01.992844 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:51:01.992844 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:51:01.992844 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:51:01.992844 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:51:01.992844 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:51:01.992844 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:51:01.992844 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:51:01.992844 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:51:01.992844 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:51:01.992844 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:51:01.992844 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:51:01.992844 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:51:01.992844 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:51:01.992844 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:51:01.993671 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:51:01.993671 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:51:01.993671 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:51:01.993671 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:51:01.993671 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:51:01.993671 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:51:01.993671 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:51:01.993671 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:51:01.993671 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:51:01.993671 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:51:01.993671 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:51:01.993671 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:51:01.993671 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 458/900] 10.222.0.39 () {56 vars in 1096 bytes} [Fri Oct 14 02:51:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:51:01.998783 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e5ed70-dea1-4500-966e-3a7ef662ef31 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:51:02.031931 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:51:02.094874 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e5ed70-dea1-4500-966e-3a7ef662ef31 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:51:02.128020 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c6b0d0-27d4-436b-bc6c-61bfb5cc7575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:51:02.131002 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c6b0d0-27d4-436b-bc6c-61bfb5cc7575 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:51:02.131002 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c6b0d0-27d4-436b-bc6c-61bfb5cc7575 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:51:02.131002 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c6b0d0-27d4-436b-bc6c-61bfb5cc7575 None admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:51:02.131002 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c6b0d0-27d4-436b-bc6c-61bfb5cc7575 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:51:02.131002 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c6b0d0-27d4-436b-bc6c-61bfb5cc7575 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:51:02.131460 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c6b0d0-27d4-436b-bc6c-61bfb5cc7575 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:51:02.139202 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 443/901] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Oct 14 02:51:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:51:02.145218 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d08c50a-6da5-4f87-b655-58c58336996f None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:51:02.241547 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d08c50a-6da5-4f87-b655-58c58336996f None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:51:02.274007 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd183c0-65a8-44a2-995f-3ad3ac84ff36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:51:02.274620 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd183c0-65a8-44a2-995f-3ad3ac84ff36 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:51:02.274900 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd183c0-65a8-44a2-995f-3ad3ac84ff36 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:51:02.275174 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd183c0-65a8-44a2-995f-3ad3ac84ff36 None admin] PATH_INFO: `/v3/users/7f8b15dd7b7241099478fd81f2fe76ad` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:51:02.276279 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd183c0-65a8-44a2-995f-3ad3ac84ff36 None admin] RBAC: Authorizing `identity:get_user(user_id=7f8b15dd7b7241099478fd81f2fe76ad)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:51:02.277706 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd183c0-65a8-44a2-995f-3ad3ac84ff36 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:51:02.279306 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 459/902] 10.222.0.39 () {56 vars in 1177 bytes} [Fri Oct 14 02:51:02 2022] GET /identity/v3/users/7f8b15dd7b7241099478fd81f2fe76ad => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:51:02.285077 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c6b0d0-27d4-436b-bc6c-61bfb5cc7575 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:51:02.376410 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:51:02.379994 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c6b0d0-27d4-436b-bc6c-61bfb5cc7575 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:51:02.411775 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668a35e9-33b4-4360-9145-a6e96f799e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:51:02.412358 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668a35e9-33b4-4360-9145-a6e96f799e1c None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:51:02.412636 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668a35e9-33b4-4360-9145-a6e96f799e1c None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:51:02.412906 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668a35e9-33b4-4360-9145-a6e96f799e1c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:51:02.476018 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668a35e9-33b4-4360-9145-a6e96f799e1c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:51:02.477436 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668a35e9-33b4-4360-9145-a6e96f799e1c None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:51:02.565390 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:51:02.569603 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 444/903] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Oct 14 02:51:02 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 14 02:51:02.575483 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd183c0-65a8-44a2-995f-3ad3ac84ff36 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:51:02.604990 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:51:02.671529 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd183c0-65a8-44a2-995f-3ad3ac84ff36 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:51:02.704534 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c30db2-e934-4247-8d76-6ba60051e278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:51:02.705139 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c30db2-e934-4247-8d76-6ba60051e278 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:51:02.705420 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c30db2-e934-4247-8d76-6ba60051e278 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:51:02.705690 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c30db2-e934-4247-8d76-6ba60051e278 None admin] PATH_INFO: `/v3/projects/987e4005e25e4aa8a8e5ff7aca88ea1c` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:51:02.706810 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c30db2-e934-4247-8d76-6ba60051e278 None admin] RBAC: Authorizing `identity:get_project(project_id=987e4005e25e4aa8a8e5ff7aca88ea1c)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:51:02.708239 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c30db2-e934-4247-8d76-6ba60051e278 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:51:02.709848 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 460/904] 10.222.0.39 () {56 vars in 1186 bytes} [Fri Oct 14 02:51:02 2022] GET /identity/v3/projects/987e4005e25e4aa8a8e5ff7aca88ea1c => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:51:02.715757 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668a35e9-33b4-4360-9145-a6e96f799e1c None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:51:02.809296 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668a35e9-33b4-4360-9145-a6e96f799e1c None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:51:02.841109 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6fb0b8-a890-406f-9ff6-dbd83c9a7d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:51:02.841688 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6fb0b8-a890-406f-9ff6-dbd83c9a7d8e None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:51:02.841992 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6fb0b8-a890-406f-9ff6-dbd83c9a7d8e None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:51:02.842262 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6fb0b8-a890-406f-9ff6-dbd83c9a7d8e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:51:02.842966 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6fb0b8-a890-406f-9ff6-dbd83c9a7d8e None admin] RBAC: Adding query filter params (role.id=9e86d182c0594e0785b72ba5f390a114, scope.project.id=987e4005e25e4aa8a8e5ff7aca88ea1c, user.id=7f8b15dd7b7241099478fd81f2fe76ad) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:51:02.843270 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6fb0b8-a890-406f-9ff6-dbd83c9a7d8e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:51:02.844692 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6fb0b8-a890-406f-9ff6-dbd83c9a7d8e None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:51:02.850187 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 445/905] 10.222.0.39 () {56 vars in 1374 bytes} [Fri Oct 14 02:51:02 2022] GET /identity/v3/role_assignments?user.id=7f8b15dd7b7241099478fd81f2fe76ad&scope.project.id=987e4005e25e4aa8a8e5ff7aca88ea1c&role.id=9e86d182c0594e0785b72ba5f390a114 => generated 608 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:51:12.699914 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=51 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:51:12.847654 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=69 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:52:15.484601 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3189ff-c370-48a1-a53b-7241d45d7e85 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:52:15.485180 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3189ff-c370-48a1-a53b-7241d45d7e85 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:52:15.485180 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3189ff-c370-48a1-a53b-7241d45d7e85 None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:52:15.486519 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 461/906] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:52:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 14 02:52:15.491713 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f380ad-65e0-4ee6-8103-e2ce3d9afdab None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:52:15.492141 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f380ad-65e0-4ee6-8103-e2ce3d9afdab None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:52:15.492263 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f380ad-65e0-4ee6-8103-e2ce3d9afdab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:52:15.509962 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-76f380ad-65e0-4ee6-8103-e2ce3d9afdab None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:52:15.511797 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-76f380ad-65e0-4ee6-8103-e2ce3d9afdab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:52:15.574733 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 446/907] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:52:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:52:15.846932 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3189ff-c370-48a1-a53b-7241d45d7e85 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:52:15.947070 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3189ff-c370-48a1-a53b-7241d45d7e85 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:52:15.980274 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930fa7c9-8578-4c7a-afee-6ed6cd73a184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:52:15.980890 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930fa7c9-8578-4c7a-afee-6ed6cd73a184 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:52:15.981168 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930fa7c9-8578-4c7a-afee-6ed6cd73a184 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:52:15.981438 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930fa7c9-8578-4c7a-afee-6ed6cd73a184 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:52:15.991152 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930fa7c9-8578-4c7a-afee-6ed6cd73a184 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:52:15.992588 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930fa7c9-8578-4c7a-afee-6ed6cd73a184 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:52:16.007443 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-930fa7c9-8578-4c7a-afee-6ed6cd73a184 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:52:16.007443 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:52:16.007443 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:52:16.007443 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:52:16.007443 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:52:16.007443 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:52:16.007443 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:52:16.007443 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:52:16.007443 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:52:16.007443 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:52:16.007443 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:52:16.007443 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:52:16.007443 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:52:16.007443 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:52:16.007443 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:52:16.007443 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:52:16.008436 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:52:16.008436 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:52:16.008436 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:52:16.008436 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:52:16.008436 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:52:16.008436 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:52:16.008436 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:52:16.008436 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:52:16.008436 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:52:16.008436 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:52:16.008436 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:52:16.008436 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:52:16.008436 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:52:16.008436 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:52:16.008436 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:52:16.008436 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:52:16.008436 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:52:16.009428 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:52:16.009428 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:52:16.009428 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:52:16.009428 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 462/908] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:52:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:52:16.014758 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f380ad-65e0-4ee6-8103-e2ce3d9afdab None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:52:16.110522 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f380ad-65e0-4ee6-8103-e2ce3d9afdab None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:52:16.138417 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:52:16.144550 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1c064e-0db1-4e25-a972-0ceadf018608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:52:16.151685 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1c064e-0db1-4e25-a972-0ceadf018608 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:52:16.151685 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1c064e-0db1-4e25-a972-0ceadf018608 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:52:16.151685 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1c064e-0db1-4e25-a972-0ceadf018608 None admin] PATH_INFO: `/v3/domains` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:52:16.151685 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1c064e-0db1-4e25-a972-0ceadf018608 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:52:16.151685 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1c064e-0db1-4e25-a972-0ceadf018608 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:52:16.151685 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1c064e-0db1-4e25-a972-0ceadf018608 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:52:16.159237 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 447/909] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Oct 14 02:52:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:52:16.164990 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930fa7c9-8578-4c7a-afee-6ed6cd73a184 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:52:16.217415 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:52:16.263976 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930fa7c9-8578-4c7a-afee-6ed6cd73a184 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:52:16.296582 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5080eb5c-b007-4b75-8cf2-ca9cec8f3ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:52:16.297691 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5080eb5c-b007-4b75-8cf2-ca9cec8f3ad2 None admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:52:16.298003 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5080eb5c-b007-4b75-8cf2-ca9cec8f3ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:52:16.298281 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5080eb5c-b007-4b75-8cf2-ca9cec8f3ad2 None admin] PATH_INFO: `/v3/users` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:52:16.298940 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5080eb5c-b007-4b75-8cf2-ca9cec8f3ad2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:52:16.300366 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5080eb5c-b007-4b75-8cf2-ca9cec8f3ad2 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:52:16.408853 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 463/910] 10.222.0.39 () {60 vars in 1132 bytes} [Fri Oct 14 02:52:16 2022] POST /identity/v3/users => generated 251 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 02:52:18.558668 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45efc6c9-9e63-4b11-8cb6-1bd17ce42f24 None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:52:18.559214 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45efc6c9-9e63-4b11-8cb6-1bd17ce42f24 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:52:18.559329 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45efc6c9-9e63-4b11-8cb6-1bd17ce42f24 None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:52:18.560586 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 448/911] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:52:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:52:18.566088 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7cd991-f20a-45af-bd50-c4d19d8e7028 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:52:18.566536 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7cd991-f20a-45af-bd50-c4d19d8e7028 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:52:18.566668 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7cd991-f20a-45af-bd50-c4d19d8e7028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:52:18.585017 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-6f7cd991-f20a-45af-bd50-c4d19d8e7028 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:52:18.586520 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-6f7cd991-f20a-45af-bd50-c4d19d8e7028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:52:18.663058 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 464/912] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:52:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:52:18.933361 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45efc6c9-9e63-4b11-8cb6-1bd17ce42f24 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:52:19.034217 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45efc6c9-9e63-4b11-8cb6-1bd17ce42f24 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:52:19.066339 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4d2ebe-d163-4c0a-ad53-9fe5f1d921c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:52:19.066959 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4d2ebe-d163-4c0a-ad53-9fe5f1d921c3 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:52:19.067239 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4d2ebe-d163-4c0a-ad53-9fe5f1d921c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:52:19.067511 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4d2ebe-d163-4c0a-ad53-9fe5f1d921c3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:52:19.077559 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4d2ebe-d163-4c0a-ad53-9fe5f1d921c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:52:19.082903 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-1e4d2ebe-d163-4c0a-ad53-9fe5f1d921c3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 14 02:52:19.082903 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:52:19.082903 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:52:19.082903 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:52:19.082903 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:52:19.082903 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:52:19.082903 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:52:19.082903 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:52:19.082903 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:52:19.082903 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:52:19.082903 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:52:19.082903 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:52:19.082903 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:52:19.082903 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:52:19.082903 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:52:19.082903 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:52:19.084020 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:52:19.084020 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:52:19.084020 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:52:19.084020 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:52:19.084020 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:52:19.084020 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:52:19.084020 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:52:19.084020 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:52:19.084020 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:52:19.084020 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:52:19.084020 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:52:19.084020 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:52:19.084020 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:52:19.084020 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:52:19.084020 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:52:19.084020 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:52:19.085075 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:52:19.085075 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:52:19.085075 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:52:19.085075 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:52:19.085075 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:52:19.085075 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:52:19.085075 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:52:19.085075 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:52:19.085075 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:52:19.085075 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:52:19.085075 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 14 02:52:19.085075 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:52:19.085075 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 449/913] 10.222.0.39 () {56 vars in 1102 bytes} [Fri Oct 14 02:52:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:52:19.090145 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7cd991-f20a-45af-bd50-c4d19d8e7028 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:52:19.128230 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:52:19.185706 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7cd991-f20a-45af-bd50-c4d19d8e7028 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:52:19.218496 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91f2047-6e98-4b29-b646-31e86cc93243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:52:19.219089 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f2047-6e98-4b29-b646-31e86cc93243 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:52:19.219365 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f2047-6e98-4b29-b646-31e86cc93243 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:52:19.219633 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f2047-6e98-4b29-b646-31e86cc93243 None admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:52:19.220281 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91f2047-6e98-4b29-b646-31e86cc93243 None admin] RBAC: Adding query filter params (name=service) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:52:19.220582 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91f2047-6e98-4b29-b646-31e86cc93243 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:52:19.221970 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91f2047-6e98-4b29-b646-31e86cc93243 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:52:19.230490 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 465/914] 10.222.0.39 () {56 vars in 1103 bytes} [Fri Oct 14 02:52:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:52:19.236906 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4d2ebe-d163-4c0a-ad53-9fe5f1d921c3 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:52:19.285919 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:52:19.333719 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4d2ebe-d163-4c0a-ad53-9fe5f1d921c3 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:52:19.366186 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c65b34-59b7-47aa-9659-3424606f5e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:52:19.366875 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c65b34-59b7-47aa-9659-3424606f5e1e None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:52:19.367160 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c65b34-59b7-47aa-9659-3424606f5e1e None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:52:19.367441 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c65b34-59b7-47aa-9659-3424606f5e1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:52:19.378356 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c65b34-59b7-47aa-9659-3424606f5e1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:52:19.379808 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c65b34-59b7-47aa-9659-3424606f5e1e None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:52:19.394117 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-b8c65b34-59b7-47aa-9659-3424606f5e1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:52:19.394117 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:52:19.394117 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:52:19.394117 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:52:19.394117 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:52:19.394117 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:52:19.394117 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:52:19.394117 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:52:19.394117 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:52:19.394117 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:52:19.394117 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:52:19.394117 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:52:19.394117 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:52:19.394117 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:52:19.394117 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:52:19.394117 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:52:19.395274 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:52:19.395274 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:52:19.395274 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:52:19.395274 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:52:19.395274 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:52:19.395274 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:52:19.395274 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:52:19.395274 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:52:19.395274 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:52:19.395274 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:52:19.395274 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:52:19.395274 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:52:19.395274 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:52:19.395274 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:52:19.395274 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:52:19.395274 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:52:19.395274 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:52:19.396204 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:52:19.396204 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:52:19.396204 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:52:19.396204 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 450/915] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:52:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:52:19.401182 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91f2047-6e98-4b29-b646-31e86cc93243 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:52:19.496591 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:52:19.499555 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91f2047-6e98-4b29-b646-31e86cc93243 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:52:19.532871 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdbfb26-5d2d-4498-b7de-1fe0a7c250a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:52:19.533467 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdbfb26-5d2d-4498-b7de-1fe0a7c250a1 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:52:19.533735 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdbfb26-5d2d-4498-b7de-1fe0a7c250a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:52:19.534035 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdbfb26-5d2d-4498-b7de-1fe0a7c250a1 None admin] PATH_INFO: `/v3/domains` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:52:19.534668 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdbfb26-5d2d-4498-b7de-1fe0a7c250a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:52:19.534968 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdbfb26-5d2d-4498-b7de-1fe0a7c250a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:52:19.536323 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdbfb26-5d2d-4498-b7de-1fe0a7c250a1 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:52:19.547786 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 466/916] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Oct 14 02:52:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:52:19.553502 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c65b34-59b7-47aa-9659-3424606f5e1e None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:52:19.651795 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c65b34-59b7-47aa-9659-3424606f5e1e None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:52:19.653274 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:52:19.685181 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4212f62-f801-4723-ab9a-f2f653fe24ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:52:19.685829 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4212f62-f801-4723-ab9a-f2f653fe24ad None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:52:19.686112 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4212f62-f801-4723-ab9a-f2f653fe24ad None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:52:19.686386 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4212f62-f801-4723-ab9a-f2f653fe24ad None admin] PATH_INFO: `/v3/users/placement` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:52:19.693534 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4212f62-f801-4723-ab9a-f2f653fe24ad None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:52:19.695009 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4212f62-f801-4723-ab9a-f2f653fe24ad None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:52:19.706229 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-f4212f62-f801-4723-ab9a-f2f653fe24ad None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 14 02:52:19.706229 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:52:19.706229 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:52:19.706229 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:52:19.706229 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:52:19.706229 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:52:19.706229 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:52:19.706229 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:52:19.706229 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:52:19.706229 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:52:19.706229 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:52:19.706229 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:52:19.706229 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 14 02:52:19.706229 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 14 02:52:19.706229 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 14 02:52:19.706229 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 14 02:52:19.707249 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:52:19.707249 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:52:19.707249 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 14 02:52:19.707249 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:52:19.707249 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 14 02:52:19.707249 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:52:19.707249 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:52:19.707249 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:52:19.707249 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:52:19.707249 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:52:19.707249 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:52:19.707249 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:52:19.707249 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:52:19.707249 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:52:19.707249 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:52:19.707249 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:52:19.707249 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:52:19.707249 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:52:19.708201 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:52:19.708201 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:52:19.708201 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 14 02:52:19.708201 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 14 02:52:19.708201 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 14 02:52:19.708201 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 14 02:52:19.708201 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 14 02:52:19.708201 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 14 02:52:19.708201 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 14 02:52:19.708201 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:52:19.708201 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 451/917] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:52:19 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:52:19.713276 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdbfb26-5d2d-4498-b7de-1fe0a7c250a1 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:52:19.812268 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdbfb26-5d2d-4498-b7de-1fe0a7c250a1 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:52:19.845144 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5507dde-4503-4abc-8d88-cc9933805d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:52:19.845793 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5507dde-4503-4abc-8d88-cc9933805d1f None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:52:19.846081 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5507dde-4503-4abc-8d88-cc9933805d1f None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:52:19.846352 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5507dde-4503-4abc-8d88-cc9933805d1f None admin] PATH_INFO: `/v3/users` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:52:19.847073 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5507dde-4503-4abc-8d88-cc9933805d1f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:52:19.847404 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5507dde-4503-4abc-8d88-cc9933805d1f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:52:19.848823 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5507dde-4503-4abc-8d88-cc9933805d1f None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:52:19.866134 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 467/918] 10.222.0.39 () {56 vars in 1143 bytes} [Fri Oct 14 02:52:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:52:19.872041 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4212f62-f801-4723-ab9a-f2f653fe24ad None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:52:19.969858 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4212f62-f801-4723-ab9a-f2f653fe24ad None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:52:20.003345 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a197eb7-f830-4a18-a1da-cdccd39223a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:52:20.003980 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a197eb7-f830-4a18-a1da-cdccd39223a8 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:52:20.004267 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a197eb7-f830-4a18-a1da-cdccd39223a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:52:20.004547 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a197eb7-f830-4a18-a1da-cdccd39223a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:52:20.015291 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a197eb7-f830-4a18-a1da-cdccd39223a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:52:20.017244 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a197eb7-f830-4a18-a1da-cdccd39223a8 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:52:20.030913 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-2a197eb7-f830-4a18-a1da-cdccd39223a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:52:20.030913 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:52:20.030913 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:52:20.030913 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:52:20.030913 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:52:20.030913 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:52:20.030913 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:52:20.030913 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:52:20.030913 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:52:20.030913 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:52:20.030913 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:52:20.030913 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:52:20.030913 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:52:20.030913 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:52:20.030913 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:52:20.030913 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:52:20.032608 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:52:20.032608 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:52:20.032608 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:52:20.032608 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:52:20.032608 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:52:20.032608 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:52:20.032608 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:52:20.032608 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:52:20.032608 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:52:20.032608 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:52:20.032608 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:52:20.032608 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:52:20.032608 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:52:20.032608 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:52:20.032608 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:52:20.032608 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:52:20.032608 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:52:20.033792 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:52:20.033792 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:52:20.033792 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:52:20.033792 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 452/919] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:52:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:52:20.038340 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5507dde-4503-4abc-8d88-cc9933805d1f None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:52:20.057268 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:52:20.135798 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5507dde-4503-4abc-8d88-cc9933805d1f None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:52:20.169973 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af02e7d-8250-4d3c-8dbf-953b1b91ffaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:52:20.170568 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af02e7d-8250-4d3c-8dbf-953b1b91ffaa None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:52:20.170847 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af02e7d-8250-4d3c-8dbf-953b1b91ffaa None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:52:20.171192 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af02e7d-8250-4d3c-8dbf-953b1b91ffaa None admin] PATH_INFO: `/v3/domains` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:52:20.171763 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af02e7d-8250-4d3c-8dbf-953b1b91ffaa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:52:20.172066 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af02e7d-8250-4d3c-8dbf-953b1b91ffaa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:52:20.173430 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af02e7d-8250-4d3c-8dbf-953b1b91ffaa None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:52:20.184433 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 468/920] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Oct 14 02:52:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:52:20.190177 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a197eb7-f830-4a18-a1da-cdccd39223a8 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:52:20.206393 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:52:20.287417 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a197eb7-f830-4a18-a1da-cdccd39223a8 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:52:20.319341 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef68972-4467-4785-8122-3598aefc268b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:52:20.319959 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef68972-4467-4785-8122-3598aefc268b None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:52:20.320240 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef68972-4467-4785-8122-3598aefc268b None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:52:20.320515 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef68972-4467-4785-8122-3598aefc268b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:52:20.327002 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef68972-4467-4785-8122-3598aefc268b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:52:20.329037 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef68972-4467-4785-8122-3598aefc268b None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:52:20.339920 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-eef68972-4467-4785-8122-3598aefc268b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:52:20.339920 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:52:20.339920 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:52:20.339920 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:52:20.339920 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:52:20.339920 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:52:20.339920 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:52:20.339920 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:52:20.339920 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:52:20.339920 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:52:20.339920 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:52:20.339920 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:52:20.339920 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 14 02:52:20.339920 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 14 02:52:20.339920 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 14 02:52:20.340935 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 14 02:52:20.340935 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:52:20.340935 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:52:20.340935 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:52:20.340935 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:52:20.340935 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:52:20.340935 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:52:20.340935 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:52:20.340935 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:52:20.340935 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:52:20.340935 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:52:20.340935 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:52:20.340935 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:52:20.340935 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:52:20.340935 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:52:20.340935 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:52:20.340935 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:52:20.341979 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 14 02:52:20.341979 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 14 02:52:20.341979 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 14 02:52:20.341979 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 14 02:52:20.341979 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 14 02:52:20.341979 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 14 02:52:20.341979 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:52:20.341979 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:52:20.341979 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 453/921] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Oct 14 02:52:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:52:20.347074 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af02e7d-8250-4d3c-8dbf-953b1b91ffaa None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:52:20.424886 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:52:20.445659 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af02e7d-8250-4d3c-8dbf-953b1b91ffaa None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:52:20.477935 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55de16e-67c6-47dc-aefb-51ed7c51a183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:52:20.478800 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55de16e-67c6-47dc-aefb-51ed7c51a183 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:52:20.479078 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55de16e-67c6-47dc-aefb-51ed7c51a183 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:52:20.479351 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55de16e-67c6-47dc-aefb-51ed7c51a183 None admin] PATH_INFO: `/v3/projects` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:52:20.480035 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55de16e-67c6-47dc-aefb-51ed7c51a183 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:52:20.480399 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55de16e-67c6-47dc-aefb-51ed7c51a183 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:52:20.481795 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55de16e-67c6-47dc-aefb-51ed7c51a183 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:52:20.493099 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 469/922] 10.222.0.39 () {56 vars in 1148 bytes} [Fri Oct 14 02:52:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:52:20.499435 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef68972-4467-4785-8122-3598aefc268b None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:52:20.572679 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:52:20.596648 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef68972-4467-4785-8122-3598aefc268b None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:52:20.629099 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7de2f8f-cd1d-4220-b270-aa03f4394e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:52:20.629891 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7de2f8f-cd1d-4220-b270-aa03f4394e52 None admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:52:20.630172 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7de2f8f-cd1d-4220-b270-aa03f4394e52 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:52:20.630446 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7de2f8f-cd1d-4220-b270-aa03f4394e52 None admin] PATH_INFO: `/v3/projects/dac9f8cd285c44e889a621b8b7fe479e/users/922ef7811aed450baff7f936011b0af1/roles/9c7226aa19e742759967212b7291782b` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:52:20.644982 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7de2f8f-cd1d-4220-b270-aa03f4394e52 None admin] RBAC: Authorizing `identity:create_grant(project_id=dac9f8cd285c44e889a621b8b7fe479e, user_id=922ef7811aed450baff7f936011b0af1, role_id=9c7226aa19e742759967212b7291782b)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:52:20.646844 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7de2f8f-cd1d-4220-b270-aa03f4394e52 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:52:20.778213 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 454/923] 10.222.0.39 () {58 vars in 1439 bytes} [Fri Oct 14 02:52:20 2022] PUT /identity/v3/projects/dac9f8cd285c44e889a621b8b7fe479e/users/922ef7811aed450baff7f936011b0af1/roles/9c7226aa19e742759967212b7291782b => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:52:22.888963 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86897ba2-4cdd-4b1d-93f8-cc4cccfc449d None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:52:22.889520 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86897ba2-4cdd-4b1d-93f8-cc4cccfc449d None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:52:22.889520 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86897ba2-4cdd-4b1d-93f8-cc4cccfc449d None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:52:22.891414 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 470/924] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:52:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:52:22.896507 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eeacd4-34d9-4807-9faf-2d462c06a9ef None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:52:22.896924 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eeacd4-34d9-4807-9faf-2d462c06a9ef None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:52:22.897051 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eeacd4-34d9-4807-9faf-2d462c06a9ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:52:22.916094 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-f4eeacd4-34d9-4807-9faf-2d462c06a9ef None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:52:22.917373 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-f4eeacd4-34d9-4807-9faf-2d462c06a9ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:52:22.982242 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 455/925] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:52:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:52:23.252215 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86897ba2-4cdd-4b1d-93f8-cc4cccfc449d None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:52:23.350610 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86897ba2-4cdd-4b1d-93f8-cc4cccfc449d None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:52:23.383649 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5103bd5f-14cd-4e34-86ba-9bd4f2ed12ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:52:23.384267 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5103bd5f-14cd-4e34-86ba-9bd4f2ed12ca None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:52:23.384545 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5103bd5f-14cd-4e34-86ba-9bd4f2ed12ca None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:52:23.384823 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5103bd5f-14cd-4e34-86ba-9bd4f2ed12ca None admin] PATH_INFO: `/v3/roles/service` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:52:23.391859 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:52:23.395521 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5103bd5f-14cd-4e34-86ba-9bd4f2ed12ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:52:23.400253 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-5103bd5f-14cd-4e34-86ba-9bd4f2ed12ca None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 14 02:52:23.400253 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:52:23.400253 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:52:23.400253 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:52:23.400253 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:52:23.400253 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:52:23.400253 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:52:23.400253 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:52:23.400253 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:52:23.400253 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:52:23.400253 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:52:23.400253 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:52:23.400253 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:52:23.400253 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:52:23.400253 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:52:23.400253 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:52:23.401401 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:52:23.401401 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:52:23.401401 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:52:23.401401 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:52:23.401401 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:52:23.401401 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:52:23.401401 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:52:23.401401 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:52:23.401401 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:52:23.401401 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:52:23.401401 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:52:23.401401 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:52:23.401401 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:52:23.401401 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:52:23.401401 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:52:23.401401 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:52:23.402558 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:52:23.402558 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:52:23.402558 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:52:23.402558 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:52:23.402558 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:52:23.402558 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:52:23.402558 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:52:23.402558 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:52:23.402558 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:52:23.402558 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:52:23.402558 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 14 02:52:23.402558 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:52:23.402558 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 471/926] 10.222.0.39 () {56 vars in 1102 bytes} [Fri Oct 14 02:52:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:52:23.407335 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eeacd4-34d9-4807-9faf-2d462c06a9ef None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:52:23.461154 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:52:23.503825 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eeacd4-34d9-4807-9faf-2d462c06a9ef None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:52:23.538608 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6564709a-15cf-4637-98d9-66a620ce8a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:52:23.540108 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6564709a-15cf-4637-98d9-66a620ce8a8f None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:52:23.540108 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6564709a-15cf-4637-98d9-66a620ce8a8f None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:52:23.540108 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6564709a-15cf-4637-98d9-66a620ce8a8f None admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:52:23.540551 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6564709a-15cf-4637-98d9-66a620ce8a8f None admin] RBAC: Adding query filter params (name=service) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:52:23.540852 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6564709a-15cf-4637-98d9-66a620ce8a8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:52:23.542245 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6564709a-15cf-4637-98d9-66a620ce8a8f None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:52:23.549920 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 456/927] 10.222.0.39 () {56 vars in 1103 bytes} [Fri Oct 14 02:52:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:52:23.555901 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5103bd5f-14cd-4e34-86ba-9bd4f2ed12ca None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:52:23.654501 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5103bd5f-14cd-4e34-86ba-9bd4f2ed12ca None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:52:23.688760 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f2754f-c69a-4ed6-be5f-ad97601672c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:52:23.689368 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f2754f-c69a-4ed6-be5f-ad97601672c3 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:52:23.689646 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f2754f-c69a-4ed6-be5f-ad97601672c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:52:23.689946 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f2754f-c69a-4ed6-be5f-ad97601672c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:52:23.699716 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f2754f-c69a-4ed6-be5f-ad97601672c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:52:23.701160 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f2754f-c69a-4ed6-be5f-ad97601672c3 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:52:23.715159 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-11f2754f-c69a-4ed6-be5f-ad97601672c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:52:23.715159 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:52:23.715159 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:52:23.715159 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:52:23.715159 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:52:23.715159 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:52:23.715159 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:52:23.715159 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:52:23.715159 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:52:23.715159 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:52:23.715159 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:52:23.715159 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:52:23.715159 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:52:23.715159 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:52:23.715159 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:52:23.715159 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:52:23.716275 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:52:23.716275 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:52:23.716275 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:52:23.716275 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:52:23.716275 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:52:23.716275 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:52:23.716275 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:52:23.716275 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:52:23.716275 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:52:23.716275 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:52:23.716275 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:52:23.716275 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:52:23.716275 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:52:23.716275 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:52:23.716275 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:52:23.716275 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:52:23.716275 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:52:23.717166 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:52:23.717166 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:52:23.717166 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:52:23.717166 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 472/928] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:52:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:52:23.721940 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6564709a-15cf-4637-98d9-66a620ce8a8f None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:52:23.819486 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6564709a-15cf-4637-98d9-66a620ce8a8f None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:52:23.834526 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:52:23.853490 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e47c08-78aa-4a6e-8939-b25295296944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:52:23.854131 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e47c08-78aa-4a6e-8939-b25295296944 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:52:23.854411 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e47c08-78aa-4a6e-8939-b25295296944 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:52:23.854685 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e47c08-78aa-4a6e-8939-b25295296944 None admin] PATH_INFO: `/v3/domains` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:52:23.855315 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e47c08-78aa-4a6e-8939-b25295296944 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:52:23.855618 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e47c08-78aa-4a6e-8939-b25295296944 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:52:23.856988 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e47c08-78aa-4a6e-8939-b25295296944 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:52:23.869298 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 457/929] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Oct 14 02:52:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:52:23.875768 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f2754f-c69a-4ed6-be5f-ad97601672c3 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:52:23.924653 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:52:23.972973 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f2754f-c69a-4ed6-be5f-ad97601672c3 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:52:24.010316 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f1bfb5-8c79-4734-869f-efc43849ae86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:52:24.010950 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f1bfb5-8c79-4734-869f-efc43849ae86 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:52:24.011235 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f1bfb5-8c79-4734-869f-efc43849ae86 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:52:24.011508 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f1bfb5-8c79-4734-869f-efc43849ae86 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:52:24.019057 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f1bfb5-8c79-4734-869f-efc43849ae86 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:52:24.020503 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f1bfb5-8c79-4734-869f-efc43849ae86 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:52:24.031958 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-55f1bfb5-8c79-4734-869f-efc43849ae86 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 14 02:52:24.031958 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:52:24.031958 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:52:24.031958 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:52:24.031958 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:52:24.031958 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:52:24.031958 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:52:24.031958 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:52:24.031958 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:52:24.031958 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:52:24.031958 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:52:24.031958 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:52:24.031958 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 14 02:52:24.031958 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 14 02:52:24.031958 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 14 02:52:24.031958 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 14 02:52:24.033166 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:52:24.033166 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:52:24.033166 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 14 02:52:24.033166 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:52:24.033166 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 14 02:52:24.033166 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:52:24.033166 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:52:24.033166 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:52:24.033166 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:52:24.033166 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:52:24.033166 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:52:24.033166 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:52:24.033166 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:52:24.033166 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:52:24.033166 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:52:24.033166 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:52:24.033166 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:52:24.033166 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:52:24.034294 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:52:24.034294 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:52:24.034294 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 14 02:52:24.034294 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 14 02:52:24.034294 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 14 02:52:24.034294 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 14 02:52:24.034294 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 14 02:52:24.034294 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 14 02:52:24.034294 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 14 02:52:24.034294 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:52:24.034294 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 473/930] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:52:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:52:24.038769 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e47c08-78aa-4a6e-8939-b25295296944 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:52:24.135770 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e47c08-78aa-4a6e-8939-b25295296944 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:52:24.169026 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481fbfa4-e858-45d4-8d0a-7bb924f19ec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:52:24.169617 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481fbfa4-e858-45d4-8d0a-7bb924f19ec3 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:52:24.169924 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481fbfa4-e858-45d4-8d0a-7bb924f19ec3 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:52:24.170198 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481fbfa4-e858-45d4-8d0a-7bb924f19ec3 None admin] PATH_INFO: `/v3/users` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:52:24.170912 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481fbfa4-e858-45d4-8d0a-7bb924f19ec3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:52:24.171216 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481fbfa4-e858-45d4-8d0a-7bb924f19ec3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:52:24.173403 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481fbfa4-e858-45d4-8d0a-7bb924f19ec3 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:52:24.189739 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 458/931] 10.222.0.39 () {56 vars in 1143 bytes} [Fri Oct 14 02:52:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:52:24.195780 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f1bfb5-8c79-4734-869f-efc43849ae86 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:52:24.294777 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f1bfb5-8c79-4734-869f-efc43849ae86 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:52:24.305473 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:52:24.327913 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3112026-ce4b-4a82-a6b4-1277b0577e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:52:24.328506 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3112026-ce4b-4a82-a6b4-1277b0577e70 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:52:24.328787 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3112026-ce4b-4a82-a6b4-1277b0577e70 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:52:24.329059 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3112026-ce4b-4a82-a6b4-1277b0577e70 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:52:24.395659 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3112026-ce4b-4a82-a6b4-1277b0577e70 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:52:24.397079 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3112026-ce4b-4a82-a6b4-1277b0577e70 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:52:24.492265 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 474/932] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Oct 14 02:52:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 14 02:52:24.498097 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481fbfa4-e858-45d4-8d0a-7bb924f19ec3 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:52:24.532320 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:52:24.596205 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481fbfa4-e858-45d4-8d0a-7bb924f19ec3 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:52:24.629562 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e4b042-ea85-4755-998a-32d7b477a0e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:52:24.630212 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e4b042-ea85-4755-998a-32d7b477a0e7 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:52:24.630496 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e4b042-ea85-4755-998a-32d7b477a0e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:52:24.630770 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e4b042-ea85-4755-998a-32d7b477a0e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:52:24.640819 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e4b042-ea85-4755-998a-32d7b477a0e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:52:24.642287 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e4b042-ea85-4755-998a-32d7b477a0e7 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:52:24.657845 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-99e4b042-ea85-4755-998a-32d7b477a0e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:52:24.657845 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:52:24.657845 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:52:24.657845 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:52:24.657845 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:52:24.657845 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:52:24.657845 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:52:24.657845 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:52:24.657845 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:52:24.657845 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:52:24.657845 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:52:24.657845 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:52:24.657845 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:52:24.657845 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:52:24.657845 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:52:24.657845 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:52:24.658790 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:52:24.658790 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:52:24.658790 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:52:24.658790 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:52:24.658790 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:52:24.658790 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:52:24.658790 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:52:24.658790 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:52:24.658790 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:52:24.658790 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:52:24.658790 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:52:24.658790 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:52:24.658790 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:52:24.658790 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:52:24.658790 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:52:24.658790 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:52:24.658790 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:52:24.659758 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:52:24.659758 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:52:24.659758 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:52:24.659758 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 459/933] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:52:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:52:24.664599 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3112026-ce4b-4a82-a6b4-1277b0577e70 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:52:24.676049 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:52:24.762348 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3112026-ce4b-4a82-a6b4-1277b0577e70 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:52:24.795926 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c03e48-64fa-4e41-8e3b-8798659286d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:52:24.796524 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c03e48-64fa-4e41-8e3b-8798659286d5 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:52:24.796808 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c03e48-64fa-4e41-8e3b-8798659286d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:52:24.797084 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c03e48-64fa-4e41-8e3b-8798659286d5 None admin] PATH_INFO: `/v3/domains` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:52:24.797808 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c03e48-64fa-4e41-8e3b-8798659286d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:52:24.798118 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c03e48-64fa-4e41-8e3b-8798659286d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:52:24.799486 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c03e48-64fa-4e41-8e3b-8798659286d5 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:52:24.810344 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 475/934] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Oct 14 02:52:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:52:24.816556 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e4b042-ea85-4755-998a-32d7b477a0e7 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:52:24.899108 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:52:24.913099 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e4b042-ea85-4755-998a-32d7b477a0e7 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:52:24.946348 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39afa8fd-787f-46d3-8880-473d28239b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:52:24.946976 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39afa8fd-787f-46d3-8880-473d28239b6c None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:52:24.947257 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39afa8fd-787f-46d3-8880-473d28239b6c None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:52:24.947535 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39afa8fd-787f-46d3-8880-473d28239b6c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:52:24.955269 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39afa8fd-787f-46d3-8880-473d28239b6c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:52:24.956706 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39afa8fd-787f-46d3-8880-473d28239b6c None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:52:24.967660 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-39afa8fd-787f-46d3-8880-473d28239b6c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:52:24.967660 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:52:24.967660 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:52:24.967660 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:52:24.967660 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:52:24.967660 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:52:24.967660 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:52:24.967660 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:52:24.967660 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:52:24.967660 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:52:24.967660 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:52:24.967660 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:52:24.967660 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 14 02:52:24.967660 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 14 02:52:24.967660 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 14 02:52:24.968742 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 14 02:52:24.968742 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:52:24.968742 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:52:24.968742 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:52:24.968742 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:52:24.968742 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:52:24.968742 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:52:24.968742 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:52:24.968742 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:52:24.968742 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:52:24.968742 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:52:24.968742 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:52:24.968742 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:52:24.968742 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:52:24.968742 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:52:24.968742 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:52:24.968742 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:52:24.969978 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 14 02:52:24.969978 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 14 02:52:24.969978 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 14 02:52:24.969978 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 14 02:52:24.969978 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 14 02:52:24.969978 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 14 02:52:24.969978 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:52:24.969978 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:52:24.969978 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 460/935] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Oct 14 02:52:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:52:24.974686 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c03e48-64fa-4e41-8e3b-8798659286d5 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:52:25.045340 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:52:25.075412 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c03e48-64fa-4e41-8e3b-8798659286d5 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:52:25.108381 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149d34f7-0a7d-466c-8249-3e0727d6d42f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:52:25.108979 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149d34f7-0a7d-466c-8249-3e0727d6d42f None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:52:25.109262 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149d34f7-0a7d-466c-8249-3e0727d6d42f None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:52:25.109538 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149d34f7-0a7d-466c-8249-3e0727d6d42f None admin] PATH_INFO: `/v3/projects` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:52:25.110273 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149d34f7-0a7d-466c-8249-3e0727d6d42f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:52:25.110572 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149d34f7-0a7d-466c-8249-3e0727d6d42f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:52:25.111993 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149d34f7-0a7d-466c-8249-3e0727d6d42f None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:52:25.124074 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 476/936] 10.222.0.39 () {56 vars in 1148 bytes} [Fri Oct 14 02:52:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:52:25.130792 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39afa8fd-787f-46d3-8880-473d28239b6c None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:52:25.229683 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39afa8fd-787f-46d3-8880-473d28239b6c None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:52:25.262341 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a334aa18-6ae3-48cd-bd34-1c2a61759f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:52:25.262947 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a334aa18-6ae3-48cd-bd34-1c2a61759f9f None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:52:25.263228 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a334aa18-6ae3-48cd-bd34-1c2a61759f9f None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:52:25.263500 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a334aa18-6ae3-48cd-bd34-1c2a61759f9f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:52:25.264211 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a334aa18-6ae3-48cd-bd34-1c2a61759f9f None admin] RBAC: Adding query filter params (role.id=9c7226aa19e742759967212b7291782b, scope.project.id=dac9f8cd285c44e889a621b8b7fe479e, user.id=922ef7811aed450baff7f936011b0af1) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:52:25.264528 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a334aa18-6ae3-48cd-bd34-1c2a61759f9f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:52:25.265955 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a334aa18-6ae3-48cd-bd34-1c2a61759f9f None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:52:25.271484 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 461/937] 10.222.0.39 () {56 vars in 1374 bytes} [Fri Oct 14 02:52:25 2022] GET /identity/v3/role_assignments?user.id=922ef7811aed450baff7f936011b0af1&scope.project.id=dac9f8cd285c44e889a621b8b7fe479e&role.id=9c7226aa19e742759967212b7291782b => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:52:27.431353 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8e5792-2c2d-467b-b931-a38f4b56a411 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:52:27.431907 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8e5792-2c2d-467b-b931-a38f4b56a411 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:52:27.431907 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8e5792-2c2d-467b-b931-a38f4b56a411 None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:52:27.433248 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 477/938] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:52:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:52:27.438340 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddaf61e-6c04-4939-86c0-d191166fcef7 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:52:27.438728 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddaf61e-6c04-4939-86c0-d191166fcef7 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:52:27.438919 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddaf61e-6c04-4939-86c0-d191166fcef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:52:27.457093 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-0ddaf61e-6c04-4939-86c0-d191166fcef7 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:52:27.458356 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-0ddaf61e-6c04-4939-86c0-d191166fcef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:52:27.460989 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:52:27.521959 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 462/939] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:52:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:52:27.794822 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8e5792-2c2d-467b-b931-a38f4b56a411 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:52:27.896046 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8e5792-2c2d-467b-b931-a38f4b56a411 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:52:27.923438 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:52:27.928916 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae33f42-c428-4492-8037-8481a56ed523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:52:27.929526 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae33f42-c428-4492-8037-8481a56ed523 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:52:27.929831 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae33f42-c428-4492-8037-8481a56ed523 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:52:27.930101 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae33f42-c428-4492-8037-8481a56ed523 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:52:27.940295 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae33f42-c428-4492-8037-8481a56ed523 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:52:27.945021 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-6ae33f42-c428-4492-8037-8481a56ed523 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:52:27.945021 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:52:27.945021 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:52:27.945021 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:52:27.945021 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:52:27.945021 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:52:27.945021 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:52:27.945021 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:52:27.945021 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:52:27.945021 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:52:27.945021 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:52:27.945021 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:52:27.945021 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:52:27.945021 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:52:27.945021 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:52:27.945021 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:52:27.946851 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:52:27.946851 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:52:27.946851 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:52:27.946851 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:52:27.946851 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:52:27.946851 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:52:27.946851 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:52:27.946851 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:52:27.946851 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:52:27.946851 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:52:27.946851 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:52:27.946851 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:52:27.946851 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:52:27.946851 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:52:27.946851 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:52:27.946851 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:52:27.947712 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:52:27.947712 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:52:27.947712 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:52:27.947712 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:52:27.947712 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:52:27.947712 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:52:27.947712 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:52:27.947712 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:52:27.947712 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:52:27.947712 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:52:27.947712 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:52:27.947712 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:52:27.947712 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 478/940] 10.222.0.39 () {56 vars in 1096 bytes} [Fri Oct 14 02:52:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:52:27.952342 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddaf61e-6c04-4939-86c0-d191166fcef7 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:52:28.049465 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddaf61e-6c04-4939-86c0-d191166fcef7 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:52:28.082095 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63563acb-c9d5-4922-8b77-40b8705351a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:52:28.082700 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63563acb-c9d5-4922-8b77-40b8705351a6 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:52:28.082978 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63563acb-c9d5-4922-8b77-40b8705351a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:52:28.083248 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63563acb-c9d5-4922-8b77-40b8705351a6 None admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:52:28.083898 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63563acb-c9d5-4922-8b77-40b8705351a6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:52:28.084198 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63563acb-c9d5-4922-8b77-40b8705351a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:52:28.085562 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63563acb-c9d5-4922-8b77-40b8705351a6 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:52:28.093171 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 463/941] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Oct 14 02:52:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:52:28.099160 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae33f42-c428-4492-8037-8481a56ed523 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:52:28.197470 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae33f42-c428-4492-8037-8481a56ed523 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:52:28.230838 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6ffe3b-1abd-4ba4-b760-1e248fae70c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:52:28.231448 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6ffe3b-1abd-4ba4-b760-1e248fae70c2 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:52:28.231726 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6ffe3b-1abd-4ba4-b760-1e248fae70c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:52:28.231995 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6ffe3b-1abd-4ba4-b760-1e248fae70c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:52:28.242269 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6ffe3b-1abd-4ba4-b760-1e248fae70c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:52:28.243712 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6ffe3b-1abd-4ba4-b760-1e248fae70c2 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:52:28.258529 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-9e6ffe3b-1abd-4ba4-b760-1e248fae70c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:52:28.258529 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:52:28.258529 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:52:28.258529 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:52:28.258529 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:52:28.258529 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:52:28.258529 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:52:28.258529 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:52:28.258529 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:52:28.258529 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:52:28.258529 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:52:28.258529 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:52:28.258529 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:52:28.258529 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:52:28.258529 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:52:28.258529 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:52:28.259602 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:52:28.259602 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:52:28.259602 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:52:28.259602 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:52:28.259602 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:52:28.259602 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:52:28.259602 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:52:28.259602 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:52:28.259602 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:52:28.259602 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:52:28.259602 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:52:28.259602 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:52:28.259602 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:52:28.259602 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:52:28.259602 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:52:28.259602 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:52:28.259602 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:52:28.260500 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:52:28.260500 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:52:28.260500 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:52:28.260500 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 479/942] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:52:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:52:28.265518 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63563acb-c9d5-4922-8b77-40b8705351a6 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:52:28.268374 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:52:28.363218 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63563acb-c9d5-4922-8b77-40b8705351a6 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:52:28.394981 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98536878-6c56-4a82-a466-6bc7c444d6a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:52:28.395576 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98536878-6c56-4a82-a466-6bc7c444d6a5 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:52:28.395859 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98536878-6c56-4a82-a466-6bc7c444d6a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:52:28.396132 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98536878-6c56-4a82-a466-6bc7c444d6a5 None admin] PATH_INFO: `/v3/domains` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:52:28.396767 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98536878-6c56-4a82-a466-6bc7c444d6a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:52:28.397073 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98536878-6c56-4a82-a466-6bc7c444d6a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:52:28.398495 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98536878-6c56-4a82-a466-6bc7c444d6a5 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:52:28.410347 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 464/943] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Oct 14 02:52:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:52:28.416109 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6ffe3b-1abd-4ba4-b760-1e248fae70c2 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:52:28.458895 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:52:28.513411 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6ffe3b-1abd-4ba4-b760-1e248fae70c2 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:52:28.545712 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bca8da-af67-49b9-99d1-aafd016a8685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:52:28.546942 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bca8da-af67-49b9-99d1-aafd016a8685 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:52:28.547219 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bca8da-af67-49b9-99d1-aafd016a8685 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:52:28.547488 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bca8da-af67-49b9-99d1-aafd016a8685 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:52:28.554459 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bca8da-af67-49b9-99d1-aafd016a8685 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:52:28.555892 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bca8da-af67-49b9-99d1-aafd016a8685 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:52:28.567331 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-81bca8da-af67-49b9-99d1-aafd016a8685 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 14 02:52:28.567331 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:52:28.567331 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:52:28.567331 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:52:28.567331 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:52:28.567331 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:52:28.567331 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:52:28.567331 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:52:28.567331 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:52:28.567331 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:52:28.567331 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:52:28.567331 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:52:28.567331 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 14 02:52:28.567331 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 14 02:52:28.567331 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 14 02:52:28.567331 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 14 02:52:28.568339 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:52:28.568339 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:52:28.568339 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 14 02:52:28.568339 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:52:28.568339 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 14 02:52:28.568339 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:52:28.568339 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:52:28.568339 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:52:28.568339 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:52:28.568339 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:52:28.568339 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:52:28.568339 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:52:28.568339 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:52:28.568339 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:52:28.568339 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:52:28.568339 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:52:28.568339 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:52:28.568339 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:52:28.569427 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:52:28.569427 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:52:28.569427 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 14 02:52:28.569427 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 14 02:52:28.569427 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 14 02:52:28.569427 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 14 02:52:28.569427 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 14 02:52:28.569427 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 14 02:52:28.569427 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 14 02:52:28.569427 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:52:28.569427 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 480/944] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:52:28 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:52:28.574858 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98536878-6c56-4a82-a466-6bc7c444d6a5 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:52:28.634482 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:52:28.670549 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98536878-6c56-4a82-a466-6bc7c444d6a5 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:52:28.703399 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f73f6b-cc95-46bb-b175-4febad1a4a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:52:28.704100 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f73f6b-cc95-46bb-b175-4febad1a4a13 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:52:28.704380 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f73f6b-cc95-46bb-b175-4febad1a4a13 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:52:28.704655 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f73f6b-cc95-46bb-b175-4febad1a4a13 None admin] PATH_INFO: `/v3/users` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:52:28.705477 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f73f6b-cc95-46bb-b175-4febad1a4a13 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:52:28.705805 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f73f6b-cc95-46bb-b175-4febad1a4a13 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:52:28.707225 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f73f6b-cc95-46bb-b175-4febad1a4a13 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:52:28.724084 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 465/945] 10.222.0.39 () {56 vars in 1143 bytes} [Fri Oct 14 02:52:28 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:52:28.730182 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bca8da-af67-49b9-99d1-aafd016a8685 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:52:28.826610 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bca8da-af67-49b9-99d1-aafd016a8685 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:52:28.828062 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:52:28.860594 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3387ea69-8f49-493a-916a-8ef1ec7cc0f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:52:28.861203 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3387ea69-8f49-493a-916a-8ef1ec7cc0f1 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:52:28.861476 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3387ea69-8f49-493a-916a-8ef1ec7cc0f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:52:28.861742 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3387ea69-8f49-493a-916a-8ef1ec7cc0f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:52:28.871469 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3387ea69-8f49-493a-916a-8ef1ec7cc0f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:52:28.872911 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3387ea69-8f49-493a-916a-8ef1ec7cc0f1 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:52:28.891583 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-3387ea69-8f49-493a-916a-8ef1ec7cc0f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:52:28.891583 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:52:28.891583 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:52:28.891583 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:52:28.891583 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:52:28.891583 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:52:28.891583 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:52:28.891583 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:52:28.891583 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:52:28.891583 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:52:28.891583 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:52:28.891583 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:52:28.891583 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:52:28.891583 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:52:28.891583 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:52:28.891583 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:52:28.892700 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:52:28.892700 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:52:28.892700 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:52:28.892700 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:52:28.892700 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:52:28.892700 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:52:28.892700 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:52:28.892700 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:52:28.892700 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:52:28.892700 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:52:28.892700 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:52:28.892700 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:52:28.892700 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:52:28.892700 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:52:28.892700 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:52:28.892700 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:52:28.892700 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:52:28.893667 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:52:28.893667 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:52:28.893667 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:52:28.893667 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 481/946] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:52:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:52:28.894292 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f73f6b-cc95-46bb-b175-4febad1a4a13 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:52:28.993056 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f73f6b-cc95-46bb-b175-4febad1a4a13 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:52:29.013037 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:52:29.027194 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8f4494-13d5-40e4-8cea-1d070bba9b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:52:29.027795 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8f4494-13d5-40e4-8cea-1d070bba9b6b None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:52:29.028074 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8f4494-13d5-40e4-8cea-1d070bba9b6b None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:52:29.028346 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8f4494-13d5-40e4-8cea-1d070bba9b6b None admin] PATH_INFO: `/v3/domains` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:52:29.028977 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8f4494-13d5-40e4-8cea-1d070bba9b6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:52:29.029279 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8f4494-13d5-40e4-8cea-1d070bba9b6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:52:29.031340 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8f4494-13d5-40e4-8cea-1d070bba9b6b None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:52:29.042221 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 466/947] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Oct 14 02:52:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 02:52:29.048087 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3387ea69-8f49-493a-916a-8ef1ec7cc0f1 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:52:29.147305 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3387ea69-8f49-493a-916a-8ef1ec7cc0f1 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:52:29.180166 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9818e298-2941-437d-a551-382e2e54c25f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:52:29.180779 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9818e298-2941-437d-a551-382e2e54c25f None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:52:29.181059 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9818e298-2941-437d-a551-382e2e54c25f None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:52:29.181330 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9818e298-2941-437d-a551-382e2e54c25f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:52:29.187803 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9818e298-2941-437d-a551-382e2e54c25f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:52:29.189239 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9818e298-2941-437d-a551-382e2e54c25f None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:52:29.200228 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-9818e298-2941-437d-a551-382e2e54c25f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:52:29.200228 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:52:29.200228 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:52:29.200228 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:52:29.200228 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:52:29.200228 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:52:29.200228 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:52:29.200228 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:52:29.200228 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:52:29.200228 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:52:29.200228 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:52:29.200228 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:52:29.200228 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 14 02:52:29.200228 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 14 02:52:29.200228 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 14 02:52:29.201301 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 14 02:52:29.201301 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:52:29.201301 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:52:29.201301 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:52:29.201301 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:52:29.201301 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:52:29.201301 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:52:29.201301 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:52:29.201301 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:52:29.201301 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:52:29.201301 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:52:29.201301 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:52:29.201301 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:52:29.201301 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:52:29.201301 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:52:29.201301 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:52:29.201301 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:52:29.202249 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 14 02:52:29.202249 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 14 02:52:29.202249 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 14 02:52:29.202249 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 14 02:52:29.202249 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 14 02:52:29.202249 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 14 02:52:29.202249 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:52:29.202249 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:52:29.202249 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 482/948] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Oct 14 02:52:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:52:29.208039 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8f4494-13d5-40e4-8cea-1d070bba9b6b None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:52:29.304067 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8f4494-13d5-40e4-8cea-1d070bba9b6b None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:52:29.337144 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdb8ff6-a821-4eba-846a-91f94eccda19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:52:29.337778 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdb8ff6-a821-4eba-846a-91f94eccda19 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:52:29.338059 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdb8ff6-a821-4eba-846a-91f94eccda19 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:52:29.338334 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdb8ff6-a821-4eba-846a-91f94eccda19 None admin] PATH_INFO: `/v3/projects` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:52:29.339013 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdb8ff6-a821-4eba-846a-91f94eccda19 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:52:29.339323 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdb8ff6-a821-4eba-846a-91f94eccda19 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:52:29.340826 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdb8ff6-a821-4eba-846a-91f94eccda19 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:52:29.352523 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 467/949] 10.222.0.39 () {56 vars in 1148 bytes} [Fri Oct 14 02:52:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:52:29.358807 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9818e298-2941-437d-a551-382e2e54c25f None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:52:29.375234 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:52:29.457707 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9818e298-2941-437d-a551-382e2e54c25f None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:52:29.490479 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffa377e-2c97-449b-9e4a-c2b942f02aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:52:29.491193 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffa377e-2c97-449b-9e4a-c2b942f02aaa None admin] REQUEST_METHOD: `PUT` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:52:29.491491 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffa377e-2c97-449b-9e4a-c2b942f02aaa None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:52:29.491761 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffa377e-2c97-449b-9e4a-c2b942f02aaa None admin] PATH_INFO: `/v3/projects/dac9f8cd285c44e889a621b8b7fe479e/users/922ef7811aed450baff7f936011b0af1/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:52:29.493607 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffa377e-2c97-449b-9e4a-c2b942f02aaa None admin] RBAC: Authorizing `identity:create_grant(project_id=dac9f8cd285c44e889a621b8b7fe479e, user_id=922ef7811aed450baff7f936011b0af1, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:52:29.495100 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffa377e-2c97-449b-9e4a-c2b942f02aaa None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:52:29.595651 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 483/950] 10.222.0.39 () {58 vars in 1439 bytes} [Fri Oct 14 02:52:29 2022] PUT /identity/v3/projects/dac9f8cd285c44e889a621b8b7fe479e/users/922ef7811aed450baff7f936011b0af1/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 02:52:31.745708 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a5db21-0f52-4e58-ad56-f032c78eda85 None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:52:31.746290 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a5db21-0f52-4e58-ad56-f032c78eda85 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:52:31.746393 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a5db21-0f52-4e58-ad56-f032c78eda85 None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:52:31.747687 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 468/951] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:52:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:52:31.753407 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6392bd6-3a5b-4fd6-ad52-b81745314d93 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:52:31.753875 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6392bd6-3a5b-4fd6-ad52-b81745314d93 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:52:31.754006 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6392bd6-3a5b-4fd6-ad52-b81745314d93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:52:31.772411 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-e6392bd6-3a5b-4fd6-ad52-b81745314d93 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:52:31.773591 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-e6392bd6-3a5b-4fd6-ad52-b81745314d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:52:31.838041 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 484/952] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:52:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:52:32.104970 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a5db21-0f52-4e58-ad56-f032c78eda85 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:52:32.147732 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:52:32.204707 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a5db21-0f52-4e58-ad56-f032c78eda85 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:52:32.237691 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23b8d5c-da4d-4f7c-bbba-2a4982358eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:52:32.238333 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23b8d5c-da4d-4f7c-bbba-2a4982358eca None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:52:32.238615 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23b8d5c-da4d-4f7c-bbba-2a4982358eca None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:52:32.238886 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23b8d5c-da4d-4f7c-bbba-2a4982358eca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:52:32.249035 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23b8d5c-da4d-4f7c-bbba-2a4982358eca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 14 02:52:32.254447 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-d23b8d5c-da4d-4f7c-bbba-2a4982358eca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:52:32.254447 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:52:32.254447 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:52:32.254447 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:52:32.254447 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:52:32.254447 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:52:32.254447 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:52:32.254447 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:52:32.254447 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:52:32.254447 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:52:32.254447 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:52:32.254447 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:52:32.254447 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 14 02:52:32.254447 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 14 02:52:32.254447 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 14 02:52:32.254447 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 14 02:52:32.258430 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 14 02:52:32.258430 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 14 02:52:32.258430 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 14 02:52:32.258430 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 14 02:52:32.258430 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:52:32.258430 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:52:32.258430 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:52:32.258430 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:52:32.258430 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:52:32.258430 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:52:32.258430 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:52:32.258430 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:52:32.258430 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:52:32.258430 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:52:32.258430 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:52:32.258430 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:52:32.259496 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:52:32.259496 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:52:32.259496 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:52:32.259496 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:52:32.259496 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 14 02:52:32.259496 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 14 02:52:32.259496 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 14 02:52:32.259496 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 14 02:52:32.259496 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 14 02:52:32.259496 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 14 02:52:32.259496 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 14 02:52:32.259496 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:52:32.259496 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 469/953] 10.222.0.39 () {56 vars in 1096 bytes} [Fri Oct 14 02:52:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:52:32.261644 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6392bd6-3a5b-4fd6-ad52-b81745314d93 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:52:32.264787 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:52:32.359309 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6392bd6-3a5b-4fd6-ad52-b81745314d93 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:52:32.391676 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e388aa90-e728-43b4-aaab-b9aeb66a8c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:52:32.392270 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388aa90-e728-43b4-aaab-b9aeb66a8c39 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:52:32.392553 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388aa90-e728-43b4-aaab-b9aeb66a8c39 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:52:32.392826 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388aa90-e728-43b4-aaab-b9aeb66a8c39 None admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:52:32.393508 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e388aa90-e728-43b4-aaab-b9aeb66a8c39 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:52:32.393921 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e388aa90-e728-43b4-aaab-b9aeb66a8c39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:52:32.399402 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e388aa90-e728-43b4-aaab-b9aeb66a8c39 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:52:32.403077 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 485/954] 10.222.0.39 () {56 vars in 1099 bytes} [Fri Oct 14 02:52:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:52:32.409498 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23b8d5c-da4d-4f7c-bbba-2a4982358eca None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:52:32.504972 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:52:32.507896 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23b8d5c-da4d-4f7c-bbba-2a4982358eca None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:52:32.540297 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e07177-cdc5-4b8a-af17-81b95ce651ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:52:32.540859 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e07177-cdc5-4b8a-af17-81b95ce651ca None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:52:32.541142 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e07177-cdc5-4b8a-af17-81b95ce651ca None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:52:32.541418 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e07177-cdc5-4b8a-af17-81b95ce651ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:52:32.551574 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e07177-cdc5-4b8a-af17-81b95ce651ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:52:32.553035 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e07177-cdc5-4b8a-af17-81b95ce651ca None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:52:32.567106 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-55e07177-cdc5-4b8a-af17-81b95ce651ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:52:32.567106 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:52:32.567106 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:52:32.567106 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:52:32.567106 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:52:32.567106 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:52:32.567106 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:52:32.567106 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:52:32.567106 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:52:32.567106 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:52:32.567106 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:52:32.567106 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:52:32.567106 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:52:32.567106 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:52:32.567106 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:52:32.567106 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:52:32.568070 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:52:32.568070 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:52:32.568070 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:52:32.568070 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:52:32.568070 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:52:32.568070 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:52:32.568070 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:52:32.568070 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:52:32.568070 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:52:32.568070 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:52:32.568070 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:52:32.568070 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:52:32.568070 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:52:32.568070 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:52:32.568070 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:52:32.568070 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:52:32.568070 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:52:32.569010 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:52:32.569010 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:52:32.569010 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:52:32.569010 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 470/955] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:52:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:52:32.574049 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e388aa90-e728-43b4-aaab-b9aeb66a8c39 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:52:32.638530 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:52:32.672692 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e388aa90-e728-43b4-aaab-b9aeb66a8c39 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:52:32.704502 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a54d8a9-fa48-48e2-8545-0933d1bdea65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:52:32.705288 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a54d8a9-fa48-48e2-8545-0933d1bdea65 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:52:32.706081 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a54d8a9-fa48-48e2-8545-0933d1bdea65 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:52:32.706359 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a54d8a9-fa48-48e2-8545-0933d1bdea65 None admin] PATH_INFO: `/v3/domains` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:52:32.707009 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a54d8a9-fa48-48e2-8545-0933d1bdea65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:52:32.707309 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a54d8a9-fa48-48e2-8545-0933d1bdea65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:52:32.708729 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a54d8a9-fa48-48e2-8545-0933d1bdea65 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:52:32.721406 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 486/956] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Oct 14 02:52:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:52:32.727292 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e07177-cdc5-4b8a-af17-81b95ce651ca None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:52:32.824789 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e07177-cdc5-4b8a-af17-81b95ce651ca None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:52:32.857811 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ba719d-ae3b-4c50-8155-23b7ea14eb54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:52:32.858430 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ba719d-ae3b-4c50-8155-23b7ea14eb54 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:52:32.858705 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ba719d-ae3b-4c50-8155-23b7ea14eb54 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:52:32.858974 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ba719d-ae3b-4c50-8155-23b7ea14eb54 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:52:32.866072 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ba719d-ae3b-4c50-8155-23b7ea14eb54 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:52:32.867482 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ba719d-ae3b-4c50-8155-23b7ea14eb54 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:52:32.878190 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-b3ba719d-ae3b-4c50-8155-23b7ea14eb54 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 14 02:52:32.878190 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:52:32.878190 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:52:32.878190 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:52:32.878190 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:52:32.878190 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:52:32.878190 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:52:32.878190 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:52:32.878190 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:52:32.878190 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:52:32.878190 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:52:32.878190 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:52:32.878190 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 14 02:52:32.878190 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 14 02:52:32.878190 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 14 02:52:32.878190 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 14 02:52:32.879358 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:52:32.879358 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:52:32.879358 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 14 02:52:32.879358 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:52:32.879358 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 14 02:52:32.879358 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 14 02:52:32.879358 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:52:32.879358 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:52:32.879358 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:52:32.879358 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:52:32.879358 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:52:32.879358 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:52:32.879358 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:52:32.879358 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:52:32.879358 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:52:32.879358 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:52:32.879358 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:52:32.879358 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:52:32.880341 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:52:32.880341 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:52:32.880341 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 14 02:52:32.880341 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 14 02:52:32.880341 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 14 02:52:32.880341 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 14 02:52:32.880341 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 14 02:52:32.880341 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 14 02:52:32.880341 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 14 02:52:32.880341 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:52:32.880341 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 471/957] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:52:32 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:52:32.885197 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a54d8a9-fa48-48e2-8545-0933d1bdea65 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:52:32.983395 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a54d8a9-fa48-48e2-8545-0933d1bdea65 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:52:33.010654 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:52:33.019822 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e756d229-5731-482c-922e-cde677880412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:52:33.020421 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e756d229-5731-482c-922e-cde677880412 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:52:33.020719 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e756d229-5731-482c-922e-cde677880412 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:52:33.020974 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e756d229-5731-482c-922e-cde677880412 None admin] PATH_INFO: `/v3/users` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:52:33.021697 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e756d229-5731-482c-922e-cde677880412 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:52:33.022033 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e756d229-5731-482c-922e-cde677880412 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:52:33.023453 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e756d229-5731-482c-922e-cde677880412 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:52:33.039907 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 487/958] 10.222.0.39 () {56 vars in 1143 bytes} [Fri Oct 14 02:52:32 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:52:33.045705 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ba719d-ae3b-4c50-8155-23b7ea14eb54 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:52:33.060586 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:52:33.144709 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ba719d-ae3b-4c50-8155-23b7ea14eb54 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:52:33.176680 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e50420-6e06-47c6-93bb-7507799f9534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:52:33.177281 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e50420-6e06-47c6-93bb-7507799f9534 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:52:33.177565 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e50420-6e06-47c6-93bb-7507799f9534 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:52:33.177914 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e50420-6e06-47c6-93bb-7507799f9534 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:52:33.244356 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e50420-6e06-47c6-93bb-7507799f9534 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:52:33.245817 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e50420-6e06-47c6-93bb-7507799f9534 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:52:33.259608 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:52:33.340924 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 472/959] 10.222.0.39 () {58 vars in 1282 bytes} [Fri Oct 14 02:52:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 14 02:52:33.346856 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e756d229-5731-482c-922e-cde677880412 None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:52:33.445301 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e756d229-5731-482c-922e-cde677880412 None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:52:33.477844 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24116f1-4408-47db-b026-197feab9175f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:52:33.478720 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24116f1-4408-47db-b026-197feab9175f None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:52:33.478997 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24116f1-4408-47db-b026-197feab9175f None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:52:33.479268 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24116f1-4408-47db-b026-197feab9175f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:52:33.489399 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24116f1-4408-47db-b026-197feab9175f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:52:33.490863 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24116f1-4408-47db-b026-197feab9175f None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:52:33.506213 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-a24116f1-4408-47db-b026-197feab9175f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:52:33.506213 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:52:33.506213 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:52:33.506213 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:52:33.506213 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:52:33.506213 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:52:33.506213 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:52:33.506213 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:52:33.506213 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:52:33.506213 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:52:33.506213 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:52:33.506213 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:52:33.506213 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 14 02:52:33.506213 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 14 02:52:33.506213 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 14 02:52:33.506213 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 14 02:52:33.507349 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:52:33.507349 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:52:33.507349 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:52:33.507349 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:52:33.507349 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:52:33.507349 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:52:33.507349 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:52:33.507349 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:52:33.507349 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:52:33.507349 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:52:33.507349 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:52:33.507349 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:52:33.507349 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:52:33.507349 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:52:33.507349 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:52:33.507349 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:52:33.507349 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 14 02:52:33.508611 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 14 02:52:33.508611 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 14 02:52:33.508611 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:52:33.508611 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 488/960] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 02:52:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 14 02:52:33.513041 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e50420-6e06-47c6-93bb-7507799f9534 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:52:33.609186 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e50420-6e06-47c6-93bb-7507799f9534 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:52:33.622536 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:52:33.643110 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82732e4c-7951-4ae0-abdc-a3ce9468582e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:52:33.643708 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82732e4c-7951-4ae0-abdc-a3ce9468582e None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:52:33.643985 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82732e4c-7951-4ae0-abdc-a3ce9468582e None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:52:33.644252 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82732e4c-7951-4ae0-abdc-a3ce9468582e None admin] PATH_INFO: `/v3/domains` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:52:33.644879 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82732e4c-7951-4ae0-abdc-a3ce9468582e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:52:33.645177 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82732e4c-7951-4ae0-abdc-a3ce9468582e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:52:33.646563 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82732e4c-7951-4ae0-abdc-a3ce9468582e None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:52:33.657198 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 473/961] 10.222.0.39 () {56 vars in 1109 bytes} [Fri Oct 14 02:52:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:52:33.662991 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24116f1-4408-47db-b026-197feab9175f None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:52:33.698965 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:52:33.763073 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24116f1-4408-47db-b026-197feab9175f None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:52:33.796687 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3066057-11b6-4f31-960b-1db14b6b0b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:52:33.797317 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3066057-11b6-4f31-960b-1db14b6b0b4d None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:52:33.797631 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3066057-11b6-4f31-960b-1db14b6b0b4d None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:52:33.797940 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3066057-11b6-4f31-960b-1db14b6b0b4d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:52:33.804511 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3066057-11b6-4f31-960b-1db14b6b0b4d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:52:33.805990 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3066057-11b6-4f31-960b-1db14b6b0b4d None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:52:33.817108 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application [None req-d3066057-11b6-4f31-960b-1db14b6b0b4d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:52:33.817108 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:52:33.817108 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:52:33.817108 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:52:33.817108 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:52:33.817108 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:52:33.817108 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:52:33.817108 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:52:33.817108 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:52:33.817108 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:52:33.817108 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:52:33.817108 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:52:33.817108 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 14 02:52:33.817108 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 14 02:52:33.817108 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 14 02:52:33.818165 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 14 02:52:33.818165 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:52:33.818165 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:52:33.818165 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:52:33.818165 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:52:33.818165 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:52:33.818165 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:52:33.818165 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:52:33.818165 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application with Lock( Oct 14 02:52:33.818165 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:52:33.818165 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:52:33.818165 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:52:33.818165 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:52:33.818165 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:52:33.818165 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:52:33.818165 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:52:33.818165 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:52:33.819142 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 14 02:52:33.819142 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 14 02:52:33.819142 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 14 02:52:33.819142 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 14 02:52:33.819142 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 14 02:52:33.819142 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 14 02:52:33.819142 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 14 02:52:33.819142 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: ERROR keystone.server.flask.application Oct 14 02:52:33.819142 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 489/962] 10.222.0.39 () {56 vars in 1111 bytes} [Fri Oct 14 02:52:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:52:33.824261 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82732e4c-7951-4ae0-abdc-a3ce9468582e None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:52:33.922952 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82732e4c-7951-4ae0-abdc-a3ce9468582e None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:52:33.954961 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac71e98-7043-4d23-b623-31b6b487f2d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:52:33.955572 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac71e98-7043-4d23-b623-31b6b487f2d3 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:52:33.955862 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac71e98-7043-4d23-b623-31b6b487f2d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:52:33.956130 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac71e98-7043-4d23-b623-31b6b487f2d3 None admin] PATH_INFO: `/v3/projects` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:52:33.957166 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac71e98-7043-4d23-b623-31b6b487f2d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:52:33.957470 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac71e98-7043-4d23-b623-31b6b487f2d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:52:33.958920 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac71e98-7043-4d23-b623-31b6b487f2d3 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:52:33.970982 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 474/963] 10.222.0.39 () {56 vars in 1148 bytes} [Fri Oct 14 02:52:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:52:33.977007 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3066057-11b6-4f31-960b-1db14b6b0b4d None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:52:34.069470 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:52:34.074975 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3066057-11b6-4f31-960b-1db14b6b0b4d None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:52:34.107263 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211f40a7-396e-4ff8-8a20-43237058b04c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:52:34.107851 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211f40a7-396e-4ff8-8a20-43237058b04c None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:52:34.108138 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211f40a7-396e-4ff8-8a20-43237058b04c None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:52:34.108946 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211f40a7-396e-4ff8-8a20-43237058b04c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:52:34.109718 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211f40a7-396e-4ff8-8a20-43237058b04c None admin] RBAC: Adding query filter params (role.id=9e86d182c0594e0785b72ba5f390a114, scope.project.id=dac9f8cd285c44e889a621b8b7fe479e, user.id=922ef7811aed450baff7f936011b0af1) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:52:34.110052 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211f40a7-396e-4ff8-8a20-43237058b04c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:52:34.111477 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211f40a7-396e-4ff8-8a20-43237058b04c None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:52:34.117648 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 490/964] 10.222.0.39 () {56 vars in 1374 bytes} [Fri Oct 14 02:52:33 2022] GET /identity/v3/role_assignments?user.id=922ef7811aed450baff7f936011b0af1&scope.project.id=dac9f8cd285c44e889a621b8b7fe479e&role.id=9e86d182c0594e0785b72ba5f390a114 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:52:36.258245 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a5d561-16c3-4fe7-8bdd-85fc7047c83a None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:52:36.258751 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a5d561-16c3-4fe7-8bdd-85fc7047c83a None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:52:36.258751 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a5d561-16c3-4fe7-8bdd-85fc7047c83a None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:52:36.260071 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 475/965] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:52:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:52:36.265950 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60df73c7-eef6-4ed4-be50-b1c25845c190 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:52:36.266368 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60df73c7-eef6-4ed4-be50-b1c25845c190 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:52:36.266502 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60df73c7-eef6-4ed4-be50-b1c25845c190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:52:36.284281 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-60df73c7-eef6-4ed4-be50-b1c25845c190 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:52:36.285455 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-60df73c7-eef6-4ed4-be50-b1c25845c190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:52:36.349765 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 491/966] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:52:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:52:36.613895 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a5d561-16c3-4fe7-8bdd-85fc7047c83a None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:52:36.644816 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:52:36.712330 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a5d561-16c3-4fe7-8bdd-85fc7047c83a None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:52:36.746078 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2898869-d07c-4861-aa10-1b944437fd0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:52:36.746701 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2898869-d07c-4861-aa10-1b944437fd0b None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:52:36.746983 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2898869-d07c-4861-aa10-1b944437fd0b None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:52:36.747254 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2898869-d07c-4861-aa10-1b944437fd0b None admin] PATH_INFO: `/v3/services/placement` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:52:36.747836 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2898869-d07c-4861-aa10-1b944437fd0b None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:52:36.749853 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2898869-d07c-4861-aa10-1b944437fd0b None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:52:36.758458 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-a2898869-d07c-4861-aa10-1b944437fd0b None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 14 02:52:36.758458 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:52:36.758458 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 14 02:52:36.758458 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 14 02:52:36.758458 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 14 02:52:36.758458 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 14 02:52:36.758458 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 14 02:52:36.758458 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:52:36.758458 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 14 02:52:36.758458 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:52:36.758458 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 02:52:36.758458 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:52:36.758458 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:52:36.758458 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:52:36.758458 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:52:36.758458 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:52:36.758458 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:52:36.760545 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:52:36.760545 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:52:36.760545 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:52:36.760545 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:52:36.760545 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:52:36.760545 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:52:36.760545 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 14 02:52:36.760545 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 14 02:52:36.760545 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 14 02:52:36.760545 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 14 02:52:36.760545 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:52:36.760545 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:52:36.760545 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:52:36.760545 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:52:36.760545 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:52:36.760545 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:52:36.761579 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:52:36.761579 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:52:36.761579 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:52:36.761579 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:52:36.761579 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:52:36.761579 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:52:36.761579 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:52:36.761579 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:52:36.761579 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:52:36.761579 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:52:36.761579 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 14 02:52:36.761579 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:52:36.761579 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 14 02:52:36.761579 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:52:36.761579 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 476/967] 10.222.0.39 () {56 vars in 1117 bytes} [Fri Oct 14 02:52:36 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:52:36.765619 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60df73c7-eef6-4ed4-be50-b1c25845c190 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:52:36.852824 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:52:36.863276 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60df73c7-eef6-4ed4-be50-b1c25845c190 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:52:36.894895 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1afcf51-a2e4-4cf1-88ba-01da9b1b3081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:52:36.895716 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1afcf51-a2e4-4cf1-88ba-01da9b1b3081 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:52:36.896001 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1afcf51-a2e4-4cf1-88ba-01da9b1b3081 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:52:36.896272 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1afcf51-a2e4-4cf1-88ba-01da9b1b3081 None admin] PATH_INFO: `/v3/services` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:52:36.896892 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1afcf51-a2e4-4cf1-88ba-01da9b1b3081 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:52:36.897199 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1afcf51-a2e4-4cf1-88ba-01da9b1b3081 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:52:36.898589 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1afcf51-a2e4-4cf1-88ba-01da9b1b3081 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:52:36.903235 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 492/968] 10.222.0.39 () {56 vars in 1116 bytes} [Fri Oct 14 02:52:36 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:52:36.909663 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2898869-d07c-4861-aa10-1b944437fd0b None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:52:37.001061 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:52:37.008925 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2898869-d07c-4861-aa10-1b944437fd0b None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:52:37.041666 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cc7906-8020-4f94-9637-bf223b6d6277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:52:37.042294 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cc7906-8020-4f94-9637-bf223b6d6277 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:52:37.042602 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cc7906-8020-4f94-9637-bf223b6d6277 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:52:37.042881 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cc7906-8020-4f94-9637-bf223b6d6277 None admin] PATH_INFO: `/v3/services` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:52:37.043505 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cc7906-8020-4f94-9637-bf223b6d6277 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:52:37.043813 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cc7906-8020-4f94-9637-bf223b6d6277 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:52:37.045176 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cc7906-8020-4f94-9637-bf223b6d6277 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:52:37.050037 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 477/969] 10.222.0.39 () {56 vars in 1116 bytes} [Fri Oct 14 02:52:36 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:52:39.172667 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d5334d-26a9-45de-93ef-6cfa8887f879 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:52:39.173219 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d5334d-26a9-45de-93ef-6cfa8887f879 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:52:39.173219 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d5334d-26a9-45de-93ef-6cfa8887f879 None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:52:39.174576 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 493/970] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:52:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:52:39.179885 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a295b249-3ce6-489d-929a-f6106c1df8e7 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:52:39.180253 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a295b249-3ce6-489d-929a-f6106c1df8e7 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:52:39.180433 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a295b249-3ce6-489d-929a-f6106c1df8e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:52:39.198094 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-a295b249-3ce6-489d-929a-f6106c1df8e7 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:52:39.199267 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-a295b249-3ce6-489d-929a-f6106c1df8e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:52:39.263756 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 478/971] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:52:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:52:39.532321 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d5334d-26a9-45de-93ef-6cfa8887f879 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:52:39.629718 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d5334d-26a9-45de-93ef-6cfa8887f879 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:52:39.662914 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dea16a-3e64-4ecb-9f12-2b9818c8632f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:52:39.663502 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dea16a-3e64-4ecb-9f12-2b9818c8632f None admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:52:39.663780 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dea16a-3e64-4ecb-9f12-2b9818c8632f None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:52:39.664054 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dea16a-3e64-4ecb-9f12-2b9818c8632f None admin] PATH_INFO: `/v3/services` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:52:39.664700 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dea16a-3e64-4ecb-9f12-2b9818c8632f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:52:39.666082 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dea16a-3e64-4ecb-9f12-2b9818c8632f None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:52:39.723048 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 494/972] 10.222.0.39 () {60 vars in 1141 bytes} [Fri Oct 14 02:52:39 2022] POST /identity/v3/services => generated 246 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:52:41.885603 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6cc8f2-078c-4a05-89a2-3e6300970856 None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:52:41.886226 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6cc8f2-078c-4a05-89a2-3e6300970856 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:52:41.886226 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6cc8f2-078c-4a05-89a2-3e6300970856 None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:52:41.887521 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 479/973] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:52:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:52:41.892721 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14490488-1114-4f62-9b6b-877751ea8750 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:52:41.893139 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14490488-1114-4f62-9b6b-877751ea8750 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:52:41.893304 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14490488-1114-4f62-9b6b-877751ea8750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:52:41.911408 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-14490488-1114-4f62-9b6b-877751ea8750 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:52:41.912599 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-14490488-1114-4f62-9b6b-877751ea8750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:52:41.922147 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:52:41.985960 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 495/974] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:52:41 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:52:42.252231 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6cc8f2-078c-4a05-89a2-3e6300970856 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:52:42.340276 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:52:42.350469 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6cc8f2-078c-4a05-89a2-3e6300970856 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:52:42.382755 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceeb5f9-8d53-4473-a8d3-dd45e9ce8afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:52:42.383467 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceeb5f9-8d53-4473-a8d3-dd45e9ce8afe None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:52:42.383758 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceeb5f9-8d53-4473-a8d3-dd45e9ce8afe None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:52:42.384035 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceeb5f9-8d53-4473-a8d3-dd45e9ce8afe None admin] PATH_INFO: `/v3/services/placement` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:52:42.384798 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dceeb5f9-8d53-4473-a8d3-dd45e9ce8afe None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:52:42.386447 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dceeb5f9-8d53-4473-a8d3-dd45e9ce8afe None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:52:42.395765 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-dceeb5f9-8d53-4473-a8d3-dd45e9ce8afe None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 14 02:52:42.395765 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:52:42.395765 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 14 02:52:42.395765 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 14 02:52:42.395765 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 14 02:52:42.395765 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 14 02:52:42.395765 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 14 02:52:42.395765 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:52:42.395765 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 14 02:52:42.395765 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:52:42.395765 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 02:52:42.395765 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:52:42.395765 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:52:42.395765 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:52:42.395765 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:52:42.395765 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:52:42.395765 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:52:42.397044 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:52:42.397044 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:52:42.397044 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:52:42.397044 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:52:42.397044 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:52:42.397044 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:52:42.397044 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 14 02:52:42.397044 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 14 02:52:42.397044 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 14 02:52:42.397044 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 14 02:52:42.397044 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:52:42.397044 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:52:42.397044 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:52:42.397044 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:52:42.397044 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:52:42.397044 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:52:42.398128 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:52:42.398128 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:52:42.398128 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:52:42.398128 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:52:42.398128 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:52:42.398128 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:52:42.398128 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:52:42.398128 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:52:42.398128 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:52:42.398128 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:52:42.398128 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 14 02:52:42.398128 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:52:42.398128 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 14 02:52:42.398128 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:52:42.398128 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 480/975] 10.222.0.39 () {56 vars in 1117 bytes} [Fri Oct 14 02:52:42 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:52:42.402950 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14490488-1114-4f62-9b6b-877751ea8750 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:52:42.497491 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14490488-1114-4f62-9b6b-877751ea8750 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:52:42.530676 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9487456c-d090-47cd-ae68-9f19c6199d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:52:42.531199 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9487456c-d090-47cd-ae68-9f19c6199d5d None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:52:42.531473 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9487456c-d090-47cd-ae68-9f19c6199d5d None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:52:42.531738 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9487456c-d090-47cd-ae68-9f19c6199d5d None admin] PATH_INFO: `/v3/services` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:52:42.532395 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9487456c-d090-47cd-ae68-9f19c6199d5d None admin] RBAC: Adding query filter params (name=placement) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:52:42.532695 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9487456c-d090-47cd-ae68-9f19c6199d5d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:52:42.534079 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9487456c-d090-47cd-ae68-9f19c6199d5d None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:52:42.538948 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 496/976] 10.222.0.39 () {56 vars in 1116 bytes} [Fri Oct 14 02:52:42 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:52:42.544837 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceeb5f9-8d53-4473-a8d3-dd45e9ce8afe None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:52:42.641343 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceeb5f9-8d53-4473-a8d3-dd45e9ce8afe None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:52:42.673411 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027d2eb2-d686-4aba-b633-1c51396b2995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:52:42.674049 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027d2eb2-d686-4aba-b633-1c51396b2995 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:52:42.674333 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027d2eb2-d686-4aba-b633-1c51396b2995 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:52:42.674603 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027d2eb2-d686-4aba-b633-1c51396b2995 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:52:42.675287 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027d2eb2-d686-4aba-b633-1c51396b2995 None admin] RBAC: Adding query filter params (interface=public, service_id=ce8d8a179b8643b2b755e7f9ea55b462, region_id=RegionOne) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:52:42.675593 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027d2eb2-d686-4aba-b633-1c51396b2995 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:52:42.676977 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027d2eb2-d686-4aba-b633-1c51396b2995 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:52:42.681787 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 481/977] 10.222.0.39 () {56 vars in 1251 bytes} [Fri Oct 14 02:52:42 2022] GET /identity/v3/endpoints?service_id=ce8d8a179b8643b2b755e7f9ea55b462&interface=public®ion_id=RegionOne => generated 194 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:52:42.687471 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9487456c-d090-47cd-ae68-9f19c6199d5d None admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:52:42.691171 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:52:42.786083 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9487456c-d090-47cd-ae68-9f19c6199d5d None admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:52:42.819076 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92678f5-8135-4b8c-afeb-df7d319729bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:52:42.819672 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92678f5-8135-4b8c-afeb-df7d319729bb None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:52:42.819955 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92678f5-8135-4b8c-afeb-df7d319729bb None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:52:42.820232 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92678f5-8135-4b8c-afeb-df7d319729bb None admin] PATH_INFO: `/v3/services` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:52:42.820829 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92678f5-8135-4b8c-afeb-df7d319729bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:52:42.822234 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92678f5-8135-4b8c-afeb-df7d319729bb None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:52:42.829571 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 497/978] 10.222.0.39 () {56 vars in 1087 bytes} [Fri Oct 14 02:52:42 2022] GET /identity/v3/services => generated 2191 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 02:52:44.940783 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c8bb9-eb03-49e8-b712-27d1517bc62e None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:52:44.941335 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c8bb9-eb03-49e8-b712-27d1517bc62e None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:52:44.941335 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c8bb9-eb03-49e8-b712-27d1517bc62e None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:52:44.942708 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 482/979] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:52:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:52:44.947904 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87543005-7eaf-4951-800f-c3d204259122 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:52:44.948309 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87543005-7eaf-4951-800f-c3d204259122 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:52:44.948464 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87543005-7eaf-4951-800f-c3d204259122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:52:44.966149 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-87543005-7eaf-4951-800f-c3d204259122 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:52:44.967364 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-87543005-7eaf-4951-800f-c3d204259122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:52:45.027812 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:52:45.033771 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 498/980] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:52:44 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 14 02:52:45.302636 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6c8bb9-eb03-49e8-b712-27d1517bc62e None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:52:45.321665 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:52:45.400082 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6c8bb9-eb03-49e8-b712-27d1517bc62e None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:52:45.432849 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44920c47-c7e5-4580-b170-5ac8d71ddca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:52:45.433463 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44920c47-c7e5-4580-b170-5ac8d71ddca2 None admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:52:45.433740 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44920c47-c7e5-4580-b170-5ac8d71ddca2 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:52:45.434046 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44920c47-c7e5-4580-b170-5ac8d71ddca2 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:52:45.434695 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44920c47-c7e5-4580-b170-5ac8d71ddca2 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:52:45.436048 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44920c47-c7e5-4580-b170-5ac8d71ddca2 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:52:45.444518 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application [None req-44920c47-c7e5-4580-b170-5ac8d71ddca2 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 14 02:52:45.444518 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:52:45.444518 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 14 02:52:45.444518 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 14 02:52:45.444518 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 14 02:52:45.444518 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 14 02:52:45.444518 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 14 02:52:45.444518 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:52:45.444518 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 14 02:52:45.444518 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:52:45.444518 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 14 02:52:45.444518 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:52:45.444518 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 14 02:52:45.444518 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 14 02:52:45.444518 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 14 02:52:45.444518 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 14 02:52:45.444518 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 14 02:52:45.445861 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 14 02:52:45.445861 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 14 02:52:45.445861 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 14 02:52:45.445861 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 14 02:52:45.445861 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 14 02:52:45.445861 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 14 02:52:45.445861 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 14 02:52:45.445861 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 14 02:52:45.445861 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 14 02:52:45.445861 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 14 02:52:45.445861 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 14 02:52:45.445861 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 14 02:52:45.445861 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 14 02:52:45.445861 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 14 02:52:45.445861 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 14 02:52:45.445861 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.get_or_create( Oct 14 02:52:45.447029 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 14 02:52:45.447029 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application with Lock( Oct 14 02:52:45.447029 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 14 02:52:45.447029 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self._enter() Oct 14 02:52:45.447029 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 14 02:52:45.447029 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 14 02:52:45.447029 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 14 02:52:45.447029 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application return self.creator() Oct 14 02:52:45.447029 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 14 02:52:45.447029 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application created_value = creator( Oct 14 02:52:45.447029 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 14 02:52:45.447029 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 14 02:52:45.447029 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 14 02:52:45.447029 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: ERROR keystone.server.flask.application Oct 14 02:52:45.447029 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 483/981] 10.222.0.39 () {56 vars in 1117 bytes} [Fri Oct 14 02:52:45 2022] GET /identity/v3/services/placement => generated 90 bytes in 144 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 14 02:52:45.452254 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87543005-7eaf-4951-800f-c3d204259122 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:52:45.547650 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87543005-7eaf-4951-800f-c3d204259122 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:52:45.580439 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9145a9-3200-4a1b-b26e-76062f69b1e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:52:45.581032 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9145a9-3200-4a1b-b26e-76062f69b1e0 None admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:52:45.581353 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9145a9-3200-4a1b-b26e-76062f69b1e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:52:45.581630 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9145a9-3200-4a1b-b26e-76062f69b1e0 None admin] PATH_INFO: `/v3/services` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:52:45.582283 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9145a9-3200-4a1b-b26e-76062f69b1e0 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 02:52:45.582589 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9145a9-3200-4a1b-b26e-76062f69b1e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:52:45.583949 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9145a9-3200-4a1b-b26e-76062f69b1e0 None admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:52:45.588867 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 499/982] 10.222.0.39 () {56 vars in 1116 bytes} [Fri Oct 14 02:52:45 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 02:52:45.595205 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44920c47-c7e5-4580-b170-5ac8d71ddca2 None admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:52:45.674644 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:52:45.691650 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44920c47-c7e5-4580-b170-5ac8d71ddca2 None admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:52:45.724387 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7343bc-3ccf-40e0-91b2-96a88f40fc00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:52:45.724980 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7343bc-3ccf-40e0-91b2-96a88f40fc00 None admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:52:45.725261 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7343bc-3ccf-40e0-91b2-96a88f40fc00 None admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:52:45.725536 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7343bc-3ccf-40e0-91b2-96a88f40fc00 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:52:45.726226 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7343bc-3ccf-40e0-91b2-96a88f40fc00 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:52:45.727589 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7343bc-3ccf-40e0-91b2-96a88f40fc00 None admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:52:45.830888 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 484/983] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 02:52:45 2022] POST /identity/v3/endpoints => generated 331 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 02:52:55.585932 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=69 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:52:55.825644 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:58:58.118449 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5936378a-d43d-48c9-88bb-865edd31c446 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:58:58.118921 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5936378a-d43d-48c9-88bb-865edd31c446 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:58:58.119167 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5936378a-d43d-48c9-88bb-865edd31c446 None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:58:58.120443 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 500/984] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:58:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:58:58.130599 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e12e08e-b19d-42ef-b10d-0b7338e1fbbe None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:58:58.130599 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e12e08e-b19d-42ef-b10d-0b7338e1fbbe None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:58:58.130599 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e12e08e-b19d-42ef-b10d-0b7338e1fbbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:58:58.217359 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-4e12e08e-b19d-42ef-b10d-0b7338e1fbbe None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:58:58.218540 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-4e12e08e-b19d-42ef-b10d-0b7338e1fbbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:58:58.323609 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 485/985] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:58:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:58:58.616890 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c6817-90bd-40e4-bc6e-5b50f63793c1 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:58:58.617575 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c6817-90bd-40e4-bc6e-5b50f63793c1 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:58:58.618064 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c6817-90bd-40e4-bc6e-5b50f63793c1 None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:58:58.619906 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 501/986] 10.222.0.39 () {54 vars in 964 bytes} [Fri Oct 14 02:58:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:58:58.626719 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05773472-a673-4b45-a66e-36df49fdd15d None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:58:58.627382 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05773472-a673-4b45-a66e-36df49fdd15d None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:58:58.627862 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05773472-a673-4b45-a66e-36df49fdd15d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:58:58.721338 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-05773472-a673-4b45-a66e-36df49fdd15d None None] MFA Rules not processed for user `b123c412ac714b06929a4bd86eca71e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:58:58.722587 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-05773472-a673-4b45-a66e-36df49fdd15d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:58:58.747110 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:58:58.808834 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 486/987] 10.222.0.39 () {58 vars in 1061 bytes} [Fri Oct 14 02:58:58 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:58:58.818271 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8c6817-90bd-40e4-bc6e-5b50f63793c1 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:58:58.820117 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-7d8c6817-90bd-40e4-bc6e-5b50f63793c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:58:58.929148 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8c6817-90bd-40e4-bc6e-5b50f63793c1 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:58:59.024107 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0691033-db0a-4cf4-a365-5b9980af08a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:58:59.024728 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0691033-db0a-4cf4-a365-5b9980af08a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:58:59.025011 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0691033-db0a-4cf4-a365-5b9980af08a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:58:59.025288 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0691033-db0a-4cf4-a365-5b9980af08a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:58:59.027677 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-f0691033-db0a-4cf4-a365-5b9980af08a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:58:59.091726 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:58:59.161364 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0691033-db0a-4cf4-a365-5b9980af08a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:58:59.163699 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0691033-db0a-4cf4-a365-5b9980af08a7 service neutron] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:58:59.188547 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 502/988] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 02:58:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:59:01.684476 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a43644c-e019-4aed-91ae-f61918ec3422 None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:59:01.684997 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a43644c-e019-4aed-91ae-f61918ec3422 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:59:01.684997 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a43644c-e019-4aed-91ae-f61918ec3422 None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:59:01.686385 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 487/989] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:59:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:59:01.691399 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09d4377-fba9-4b07-8062-f57d26d48a4b None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:59:01.691777 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09d4377-fba9-4b07-8062-f57d26d48a4b None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:59:01.691935 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09d4377-fba9-4b07-8062-f57d26d48a4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:59:01.761456 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-b09d4377-fba9-4b07-8062-f57d26d48a4b None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:59:01.762604 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-b09d4377-fba9-4b07-8062-f57d26d48a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:59:01.856755 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 503/990] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:59:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:59:02.139021 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a43644c-e019-4aed-91ae-f61918ec3422 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:59:02.140557 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-3a43644c-e019-4aed-91ae-f61918ec3422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:59:02.241890 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a43644c-e019-4aed-91ae-f61918ec3422 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:59:02.310451 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:59:02.312708 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485e638f-da82-4706-99a1-6fa9f39139f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:59:02.313303 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e638f-da82-4706-99a1-6fa9f39139f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:59:02.313582 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e638f-da82-4706-99a1-6fa9f39139f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:59:02.313881 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e638f-da82-4706-99a1-6fa9f39139f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:59:02.315452 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-485e638f-da82-4706-99a1-6fa9f39139f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:59:02.419923 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485e638f-da82-4706-99a1-6fa9f39139f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:59:02.421277 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485e638f-da82-4706-99a1-6fa9f39139f7 service neutron] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:59:02.439432 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 488/991] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 02:59:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:59:05.130700 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e7f496-8613-4c17-83f5-357d5993fdee None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:59:05.142248 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e7f496-8613-4c17-83f5-357d5993fdee None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:59:05.142248 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e7f496-8613-4c17-83f5-357d5993fdee None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:59:05.142248 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 504/992] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:59:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:59:05.142504 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54627593-1b65-4ef8-8e0b-63ffeae8a254 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:59:05.142504 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54627593-1b65-4ef8-8e0b-63ffeae8a254 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:59:05.142504 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54627593-1b65-4ef8-8e0b-63ffeae8a254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:59:05.204592 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-54627593-1b65-4ef8-8e0b-63ffeae8a254 None None] MFA Rules not processed for user `2d06bb982e494934920981820968bc00`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:59:05.205854 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-54627593-1b65-4ef8-8e0b-63ffeae8a254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:59:05.256651 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:59:05.300982 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 489/993] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:59:05 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 02:59:05.585107 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e7f496-8613-4c17-83f5-357d5993fdee None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:59:05.586672 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-40e7f496-8613-4c17-83f5-357d5993fdee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:59:05.596327 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:59:05.686604 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e7f496-8613-4c17-83f5-357d5993fdee None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:59:05.758049 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136cdbf1-697b-497e-aa66-b590165b8fc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:59:05.758642 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136cdbf1-697b-497e-aa66-b590165b8fc9 service neutron] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:59:05.758915 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136cdbf1-697b-497e-aa66-b590165b8fc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:59:05.759193 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136cdbf1-697b-497e-aa66-b590165b8fc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:59:05.761358 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-136cdbf1-697b-497e-aa66-b590165b8fc9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:59:05.820322 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:59:05.865649 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136cdbf1-697b-497e-aa66-b590165b8fc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:59:05.867024 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136cdbf1-697b-497e-aa66-b590165b8fc9 service neutron] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:59:05.885233 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 505/994] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 02:59:05 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:59:08.733065 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1249529c-ec1b-4751-b992-c2ca4e5be1d3 None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:59:08.733887 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1249529c-ec1b-4751-b992-c2ca4e5be1d3 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:59:08.733985 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1249529c-ec1b-4751-b992-c2ca4e5be1d3 None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:59:08.735271 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 490/995] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:59:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:59:08.740419 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f147e00-b09e-4e68-88a4-e0e2849c7a00 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:59:08.740849 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f147e00-b09e-4e68-88a4-e0e2849c7a00 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:59:08.740956 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f147e00-b09e-4e68-88a4-e0e2849c7a00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:59:08.804906 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-1f147e00-b09e-4e68-88a4-e0e2849c7a00 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:59:08.806133 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-1f147e00-b09e-4e68-88a4-e0e2849c7a00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:59:08.899498 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 506/996] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:59:08 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:59:09.183077 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1249529c-ec1b-4751-b992-c2ca4e5be1d3 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:59:09.184738 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-1249529c-ec1b-4751-b992-c2ca4e5be1d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:59:09.285015 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1249529c-ec1b-4751-b992-c2ca4e5be1d3 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:59:09.357575 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dca431-618a-47cb-baf7-566bfac0948d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:59:09.358470 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dca431-618a-47cb-baf7-566bfac0948d service neutron] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:59:09.358753 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dca431-618a-47cb-baf7-566bfac0948d service neutron] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:59:09.359062 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dca431-618a-47cb-baf7-566bfac0948d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:59:09.360655 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-11dca431-618a-47cb-baf7-566bfac0948d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:59:09.368059 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:59:09.469271 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dca431-618a-47cb-baf7-566bfac0948d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:59:09.470738 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dca431-618a-47cb-baf7-566bfac0948d service neutron] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:59:09.488511 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 491/997] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 02:59:09 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:59:12.672569 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8501127f-667d-4dfb-97fa-b565fa446e73 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:59:12.673163 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8501127f-667d-4dfb-97fa-b565fa446e73 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:59:12.673163 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8501127f-667d-4dfb-97fa-b565fa446e73 None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:59:12.674514 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 507/998] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:59:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:59:12.679752 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caa92d7-8436-4952-863e-d78faba6e823 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:59:12.680172 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caa92d7-8436-4952-863e-d78faba6e823 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:59:12.680284 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caa92d7-8436-4952-863e-d78faba6e823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:59:12.749117 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-5caa92d7-8436-4952-863e-d78faba6e823 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:59:12.750415 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-5caa92d7-8436-4952-863e-d78faba6e823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:59:12.812664 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:59:12.854829 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 492/999] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:59:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:59:13.142400 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8501127f-667d-4dfb-97fa-b565fa446e73 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:59:13.143935 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-8501127f-667d-4dfb-97fa-b565fa446e73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:59:13.165372 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:59:13.253788 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8501127f-667d-4dfb-97fa-b565fa446e73 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:59:13.330748 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c605e16-a7ee-4326-8480-3dd8b04d7a2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:59:13.331357 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c605e16-a7ee-4326-8480-3dd8b04d7a2a service neutron] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:59:13.331640 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c605e16-a7ee-4326-8480-3dd8b04d7a2a service neutron] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:59:13.331917 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c605e16-a7ee-4326-8480-3dd8b04d7a2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:59:13.333528 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-0c605e16-a7ee-4326-8480-3dd8b04d7a2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:59:13.404746 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:59:13.442721 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c605e16-a7ee-4326-8480-3dd8b04d7a2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:59:13.444228 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c605e16-a7ee-4326-8480-3dd8b04d7a2a service neutron] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:59:13.461674 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 508/1000] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 02:59:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:59:15.666828 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725c218a-0b20-4131-8d96-d73f79febd07 None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:59:15.667382 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725c218a-0b20-4131-8d96-d73f79febd07 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:59:15.667382 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725c218a-0b20-4131-8d96-d73f79febd07 None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:59:15.668744 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 493/1001] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:59:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:59:15.674210 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee345fd-4f41-45dd-b173-b123de64ff0d None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:59:15.674636 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee345fd-4f41-45dd-b173-b123de64ff0d None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:59:15.674775 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee345fd-4f41-45dd-b173-b123de64ff0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:59:15.740532 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-1ee345fd-4f41-45dd-b173-b123de64ff0d None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:59:15.741723 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-1ee345fd-4f41-45dd-b173-b123de64ff0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:59:15.847213 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 509/1002] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:59:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:59:16.127444 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725c218a-0b20-4131-8d96-d73f79febd07 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:59:16.135598 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-725c218a-0b20-4131-8d96-d73f79febd07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:59:16.232464 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725c218a-0b20-4131-8d96-d73f79febd07 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:59:16.305626 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213d60fc-04ee-4321-95e4-f271c5ff79a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:59:16.306759 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d60fc-04ee-4321-95e4-f271c5ff79a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:59:16.307037 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d60fc-04ee-4321-95e4-f271c5ff79a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:59:16.307306 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d60fc-04ee-4321-95e4-f271c5ff79a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:59:16.308895 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-213d60fc-04ee-4321-95e4-f271c5ff79a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:59:16.326220 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:59:16.418716 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213d60fc-04ee-4321-95e4-f271c5ff79a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:59:16.420071 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213d60fc-04ee-4321-95e4-f271c5ff79a0 service neutron] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:59:16.438600 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 494/1003] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 02:59:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:59:18.623749 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e742d71-99d8-43bf-8bff-e0e5f81efed5 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:59:18.624229 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e742d71-99d8-43bf-8bff-e0e5f81efed5 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:59:18.624229 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e742d71-99d8-43bf-8bff-e0e5f81efed5 None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:59:18.625549 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 510/1004] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:59:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:59:18.631100 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81db1706-546b-4de8-b346-744fbf0cfee8 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:59:18.631508 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81db1706-546b-4de8-b346-744fbf0cfee8 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:59:18.631691 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81db1706-546b-4de8-b346-744fbf0cfee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:59:18.696347 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-81db1706-546b-4de8-b346-744fbf0cfee8 None None] MFA Rules not processed for user `2d06bb982e494934920981820968bc00`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:59:18.698076 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-81db1706-546b-4de8-b346-744fbf0cfee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:59:18.763922 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:59:18.795942 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 495/1005] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:59:18 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:59:19.100249 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c47420d-e50f-40fe-a4f5-3bcaeae44e45 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:59:19.100617 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c47420d-e50f-40fe-a4f5-3bcaeae44e45 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:59:19.100962 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c47420d-e50f-40fe-a4f5-3bcaeae44e45 None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:59:19.102423 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 511/1006] 10.222.0.39 () {54 vars in 964 bytes} [Fri Oct 14 02:59:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:59:19.109335 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ffb558-99f9-43e6-80c9-69d2f37047b0 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:59:19.109627 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ffb558-99f9-43e6-80c9-69d2f37047b0 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:59:19.109950 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ffb558-99f9-43e6-80c9-69d2f37047b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:59:19.191514 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-01ffb558-99f9-43e6-80c9-69d2f37047b0 None None] MFA Rules not processed for user `b123c412ac714b06929a4bd86eca71e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:59:19.192741 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-01ffb558-99f9-43e6-80c9-69d2f37047b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:59:19.271044 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 496/1007] 10.222.0.39 () {58 vars in 1061 bytes} [Fri Oct 14 02:59:19 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:59:19.280205 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c47420d-e50f-40fe-a4f5-3bcaeae44e45 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:59:19.281984 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-1c47420d-e50f-40fe-a4f5-3bcaeae44e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:59:19.313400 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:59:19.387482 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c47420d-e50f-40fe-a4f5-3bcaeae44e45 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:59:19.460711 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ede0a5-6b83-42d1-bae5-ba317497c375 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:59:19.461315 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ede0a5-6b83-42d1-bae5-ba317497c375 service neutron] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:59:19.461600 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ede0a5-6b83-42d1-bae5-ba317497c375 service neutron] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:59:19.461912 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ede0a5-6b83-42d1-bae5-ba317497c375 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:59:19.463535 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-01ede0a5-6b83-42d1-bae5-ba317497c375 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:59:19.547169 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:59:19.580373 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ede0a5-6b83-42d1-bae5-ba317497c375 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:59:19.582638 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ede0a5-6b83-42d1-bae5-ba317497c375 service neutron] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:59:19.602810 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 512/1008] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 02:59:19 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:59:22.518998 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc620c2-bf52-4483-964c-28bafba530ba None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:59:22.519605 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc620c2-bf52-4483-964c-28bafba530ba None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:59:22.519605 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc620c2-bf52-4483-964c-28bafba530ba None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:59:22.520899 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 497/1009] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:59:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:59:22.526662 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bda3be-5287-47ef-ac74-d07a33d851a5 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:59:22.527045 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bda3be-5287-47ef-ac74-d07a33d851a5 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:59:22.527215 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bda3be-5287-47ef-ac74-d07a33d851a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:59:22.591920 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-89bda3be-5287-47ef-ac74-d07a33d851a5 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:59:22.593126 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-89bda3be-5287-47ef-ac74-d07a33d851a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:59:22.686053 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 513/1010] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:59:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:59:22.967725 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc620c2-bf52-4483-964c-28bafba530ba None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:59:22.969246 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-bdc620c2-bf52-4483-964c-28bafba530ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:59:23.037367 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:59:23.069980 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc620c2-bf52-4483-964c-28bafba530ba None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:59:23.142244 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be68029a-01e3-41c2-926e-08541c170939 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:59:23.142850 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be68029a-01e3-41c2-926e-08541c170939 service neutron] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:59:23.143130 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be68029a-01e3-41c2-926e-08541c170939 service neutron] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:59:23.143402 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be68029a-01e3-41c2-926e-08541c170939 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:59:23.144995 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-be68029a-01e3-41c2-926e-08541c170939 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:59:23.253715 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be68029a-01e3-41c2-926e-08541c170939 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:59:23.255152 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be68029a-01e3-41c2-926e-08541c170939 service neutron] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:59:23.264908 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:59:23.273217 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 498/1011] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 02:59:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:59:25.433628 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802f6864-dffa-4c9f-94d1-6eba92730581 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:59:25.434185 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802f6864-dffa-4c9f-94d1-6eba92730581 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:59:25.434304 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802f6864-dffa-4c9f-94d1-6eba92730581 None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:59:25.435581 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 514/1012] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:59:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:59:25.440658 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238b68f1-6e6b-4338-a405-ffe197c0b773 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:59:25.441105 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238b68f1-6e6b-4338-a405-ffe197c0b773 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:59:25.441203 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238b68f1-6e6b-4338-a405-ffe197c0b773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:59:25.505663 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-238b68f1-6e6b-4338-a405-ffe197c0b773 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:59:25.506880 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-238b68f1-6e6b-4338-a405-ffe197c0b773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:59:25.599382 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 499/1013] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:59:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:59:25.877644 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802f6864-dffa-4c9f-94d1-6eba92730581 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:59:25.879215 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-802f6864-dffa-4c9f-94d1-6eba92730581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:59:25.918654 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:59:25.979791 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802f6864-dffa-4c9f-94d1-6eba92730581 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:59:26.051883 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0142e41-b5c8-4bbe-8204-b13111b43425 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:59:26.052479 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0142e41-b5c8-4bbe-8204-b13111b43425 service neutron] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:59:26.052753 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0142e41-b5c8-4bbe-8204-b13111b43425 service neutron] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:59:26.053019 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0142e41-b5c8-4bbe-8204-b13111b43425 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:59:26.054643 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-c0142e41-b5c8-4bbe-8204-b13111b43425 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:59:26.137316 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:59:26.162143 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0142e41-b5c8-4bbe-8204-b13111b43425 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:59:26.163568 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0142e41-b5c8-4bbe-8204-b13111b43425 service neutron] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:59:26.183345 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 515/1014] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 02:59:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:59:28.972936 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed533eb-7ff0-4573-b913-5c9ac9af56a2 None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:59:28.973498 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed533eb-7ff0-4573-b913-5c9ac9af56a2 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:59:28.973498 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed533eb-7ff0-4573-b913-5c9ac9af56a2 None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:59:28.974897 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 500/1015] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:59:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:59:28.980021 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8ac878-ad81-4992-a331-234efd063269 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:59:28.980425 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8ac878-ad81-4992-a331-234efd063269 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:59:28.980562 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8ac878-ad81-4992-a331-234efd063269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:59:29.045069 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-dd8ac878-ad81-4992-a331-234efd063269 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:59:29.046280 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-dd8ac878-ad81-4992-a331-234efd063269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:59:29.140280 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 516/1016] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:59:28 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:59:29.421914 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed533eb-7ff0-4573-b913-5c9ac9af56a2 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:59:29.423443 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-9ed533eb-7ff0-4573-b913-5c9ac9af56a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:59:29.498010 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:59:29.524181 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed533eb-7ff0-4573-b913-5c9ac9af56a2 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:59:29.596316 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155e84e4-b9e4-48a2-a403-9a23248f134d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:59:29.596918 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155e84e4-b9e4-48a2-a403-9a23248f134d service neutron] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:59:29.597199 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155e84e4-b9e4-48a2-a403-9a23248f134d service neutron] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:59:29.597478 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155e84e4-b9e4-48a2-a403-9a23248f134d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:59:29.599082 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-155e84e4-b9e4-48a2-a403-9a23248f134d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:59:29.706296 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155e84e4-b9e4-48a2-a403-9a23248f134d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:59:29.707783 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155e84e4-b9e4-48a2-a403-9a23248f134d service neutron] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:59:29.725915 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 501/1017] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 02:59:29 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:59:33.996588 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950dfdc1-e48b-4090-97d0-f87535c30c7c None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:59:33.997181 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950dfdc1-e48b-4090-97d0-f87535c30c7c None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:59:33.997181 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950dfdc1-e48b-4090-97d0-f87535c30c7c None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:59:33.998536 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 517/1018] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:59:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:59:34.003782 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcdeb9e-30de-473c-8a3a-00453e2060d3 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:59:34.004248 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcdeb9e-30de-473c-8a3a-00453e2060d3 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:59:34.004336 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcdeb9e-30de-473c-8a3a-00453e2060d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:59:34.010602 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:59:34.075020 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-8dcdeb9e-30de-473c-8a3a-00453e2060d3 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:59:34.076235 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-8dcdeb9e-30de-473c-8a3a-00453e2060d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:59:34.173432 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 502/1019] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:59:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:59:34.457491 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950dfdc1-e48b-4090-97d0-f87535c30c7c None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:59:34.459037 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-950dfdc1-e48b-4090-97d0-f87535c30c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:59:34.507585 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:59:34.562206 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950dfdc1-e48b-4090-97d0-f87535c30c7c None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:59:34.633893 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7363dac2-bc9d-479b-bace-645b5ad3836b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:59:34.634495 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7363dac2-bc9d-479b-bace-645b5ad3836b service neutron] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:59:34.634793 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7363dac2-bc9d-479b-bace-645b5ad3836b service neutron] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:59:34.635042 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7363dac2-bc9d-479b-bace-645b5ad3836b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:59:34.636613 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-7363dac2-bc9d-479b-bace-645b5ad3836b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:59:34.728000 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:59:34.743835 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7363dac2-bc9d-479b-bace-645b5ad3836b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:59:34.745266 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7363dac2-bc9d-479b-bace-645b5ad3836b service neutron] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:59:34.763134 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 518/1020] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 02:59:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:59:37.834559 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f73959-05ce-4b29-b14f-b45443cc57fe None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:59:37.835051 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f73959-05ce-4b29-b14f-b45443cc57fe None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:59:37.835157 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f73959-05ce-4b29-b14f-b45443cc57fe None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:59:37.836434 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 503/1021] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:59:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:59:37.842581 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d37c91-c1f9-4858-b57c-7f9cbb398599 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:59:37.843022 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d37c91-c1f9-4858-b57c-7f9cbb398599 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:59:37.843131 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d37c91-c1f9-4858-b57c-7f9cbb398599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:59:37.913637 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-80d37c91-c1f9-4858-b57c-7f9cbb398599 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:59:37.915543 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-80d37c91-c1f9-4858-b57c-7f9cbb398599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:59:38.013287 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 519/1022] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:59:37 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:59:38.302910 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f73959-05ce-4b29-b14f-b45443cc57fe None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:59:38.304434 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-67f73959-05ce-4b29-b14f-b45443cc57fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:59:38.392249 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:59:38.409438 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f73959-05ce-4b29-b14f-b45443cc57fe None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:59:38.485924 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a65b725-b072-475f-8527-4f8e3d9f71ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:59:38.486581 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a65b725-b072-475f-8527-4f8e3d9f71ea service neutron] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:59:38.486848 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a65b725-b072-475f-8527-4f8e3d9f71ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:59:38.487115 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a65b725-b072-475f-8527-4f8e3d9f71ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:59:38.488828 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-3a65b725-b072-475f-8527-4f8e3d9f71ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:59:38.616631 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a65b725-b072-475f-8527-4f8e3d9f71ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:59:38.618618 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a65b725-b072-475f-8527-4f8e3d9f71ea service neutron] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:59:38.647121 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 504/1023] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 02:59:38 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 02:59:42.991003 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22692f16-1a89-4316-abbc-ac5e3eddc5df None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:59:42.991548 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22692f16-1a89-4316-abbc-ac5e3eddc5df None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:59:42.991548 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22692f16-1a89-4316-abbc-ac5e3eddc5df None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:59:42.993437 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 520/1024] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:59:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:59:42.999073 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6180eb5a-11b9-4905-a4e8-ab160fafbf12 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:59:42.999501 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6180eb5a-11b9-4905-a4e8-ab160fafbf12 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:59:42.999621 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6180eb5a-11b9-4905-a4e8-ab160fafbf12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:59:43.017803 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:59:43.072641 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-6180eb5a-11b9-4905-a4e8-ab160fafbf12 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:59:43.073960 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-6180eb5a-11b9-4905-a4e8-ab160fafbf12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:59:43.171268 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 505/1025] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:59:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:59:43.453641 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22692f16-1a89-4316-abbc-ac5e3eddc5df None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:59:43.455200 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-22692f16-1a89-4316-abbc-ac5e3eddc5df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:59:43.511488 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:59:43.560570 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22692f16-1a89-4316-abbc-ac5e3eddc5df None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:59:43.636989 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cca33c8-361a-4afa-afb8-f849574c2406 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:59:43.637593 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cca33c8-361a-4afa-afb8-f849574c2406 service neutron] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:59:43.637901 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cca33c8-361a-4afa-afb8-f849574c2406 service neutron] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:59:43.638173 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cca33c8-361a-4afa-afb8-f849574c2406 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:59:43.639764 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-1cca33c8-361a-4afa-afb8-f849574c2406 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:59:43.747383 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:59:43.750890 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cca33c8-361a-4afa-afb8-f849574c2406 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:59:43.752409 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cca33c8-361a-4afa-afb8-f849574c2406 service neutron] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:59:43.770197 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 521/1026] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 02:59:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 02:59:48.615130 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67719b34-de59-4275-83c8-1bd632d9b2ec None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:59:48.615569 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67719b34-de59-4275-83c8-1bd632d9b2ec None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:59:48.615713 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67719b34-de59-4275-83c8-1bd632d9b2ec None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:59:48.617607 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 506/1027] 10.222.0.39 () {54 vars in 921 bytes} [Fri Oct 14 02:59:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:59:48.622818 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00e7636-5890-484a-a507-923284cdfd62 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:59:48.623195 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00e7636-5890-484a-a507-923284cdfd62 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:59:48.623459 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00e7636-5890-484a-a507-923284cdfd62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:59:48.726008 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-d00e7636-5890-484a-a507-923284cdfd62 None None] MFA Rules not processed for user `922ef7811aed450baff7f936011b0af1`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:59:48.727415 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-d00e7636-5890-484a-a507-923284cdfd62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:59:48.914823 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 522/1028] 10.222.0.39 () {58 vars in 1018 bytes} [Fri Oct 14 02:59:48 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:59:52.938034 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b71be34-cf6f-4c1f-a019-70b0f1331f90 None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:59:52.938604 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b71be34-cf6f-4c1f-a019-70b0f1331f90 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:59:52.938604 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b71be34-cf6f-4c1f-a019-70b0f1331f90 None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:59:52.940355 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 507/1029] 10.222.0.39 () {54 vars in 921 bytes} [Fri Oct 14 02:59:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:59:52.945984 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fde552-50f9-42ed-9359-5349db7c7e4e None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:59:52.946406 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fde552-50f9-42ed-9359-5349db7c7e4e None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:59:52.946611 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fde552-50f9-42ed-9359-5349db7c7e4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:59:53.010608 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:59:53.035075 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-02fde552-50f9-42ed-9359-5349db7c7e4e None None] MFA Rules not processed for user `922ef7811aed450baff7f936011b0af1`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:59:53.036645 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-02fde552-50f9-42ed-9359-5349db7c7e4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:59:53.149554 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 523/1030] 10.222.0.39 () {58 vars in 1018 bytes} [Fri Oct 14 02:59:52 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:59:53.169641 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=59 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:59:54.665844 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea24008-6224-4c60-8631-57f960f874e2 None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:59:54.666473 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea24008-6224-4c60-8631-57f960f874e2 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:59:54.666473 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea24008-6224-4c60-8631-57f960f874e2 None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:59:54.667733 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 508/1031] 10.222.0.39 () {54 vars in 921 bytes} [Fri Oct 14 02:59:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:59:54.672696 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ea2c5c-2b15-4861-8a97-bbfac94e774e None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:59:54.673150 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ea2c5c-2b15-4861-8a97-bbfac94e774e None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:59:54.673260 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ea2c5c-2b15-4861-8a97-bbfac94e774e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:59:54.753397 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-b1ea2c5c-2b15-4861-8a97-bbfac94e774e None None] MFA Rules not processed for user `922ef7811aed450baff7f936011b0af1`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:59:54.754613 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-b1ea2c5c-2b15-4861-8a97-bbfac94e774e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:59:54.798917 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:59:54.864665 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 524/1032] 10.222.0.39 () {58 vars in 1018 bytes} [Fri Oct 14 02:59:54 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 02:59:58.428006 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d58d2e-eef9-45ee-9bb9-add94fbcdb4b None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:59:58.428935 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d58d2e-eef9-45ee-9bb9-add94fbcdb4b None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:59:58.429369 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d58d2e-eef9-45ee-9bb9-add94fbcdb4b None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:59:58.433976 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 509/1033] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 02:59:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:59:58.437264 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c3cab3-6c25-4b15-9db3-6cacd6df1a1f None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:59:58.438430 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c3cab3-6c25-4b15-9db3-6cacd6df1a1f None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:59:58.438756 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c3cab3-6c25-4b15-9db3-6cacd6df1a1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:59:58.510269 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-e2c3cab3-6c25-4b15-9db3-6cacd6df1a1f None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:59:58.511463 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-e2c3cab3-6c25-4b15-9db3-6cacd6df1a1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:59:58.611842 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 525/1034] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 02:59:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:59:58.922918 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fd34b1-4c39-4e94-a50f-6978d1463609 None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:59:58.923730 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fd34b1-4c39-4e94-a50f-6978d1463609 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:59:58.924534 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fd34b1-4c39-4e94-a50f-6978d1463609 None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:59:58.928672 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 510/1035] 10.222.0.39 () {54 vars in 961 bytes} [Fri Oct 14 02:59:58 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 02:59:58.937903 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60568a88-5630-4cab-b88d-a6b03f323a06 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:59:58.938475 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60568a88-5630-4cab-b88d-a6b03f323a06 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:59:58.943539 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60568a88-5630-4cab-b88d-a6b03f323a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:59:58.962662 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:59:59.034457 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-60568a88-5630-4cab-b88d-a6b03f323a06 None None] MFA Rules not processed for user `a6b07bf0faca47b28b50990d1720853f`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 02:59:59.035647 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-60568a88-5630-4cab-b88d-a6b03f323a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:59:59.155073 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 526/1036] 10.222.0.39 () {58 vars in 1058 bytes} [Fri Oct 14 02:59:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 02:59:59.164312 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fd34b1-4c39-4e94-a50f-6978d1463609 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 02:59:59.165839 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-78fd34b1-4c39-4e94-a50f-6978d1463609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:59:59.317586 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fd34b1-4c39-4e94-a50f-6978d1463609 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 02:59:59.394928 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 02:59:59.437505 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bc766a-6fa5-4e70-a4ff-4519e55dcf9b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b07bf0faca47b28b50990d1720853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 02:59:59.438182 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bc766a-6fa5-4e70-a4ff-4519e55dcf9b service nova] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 02:59:59.438416 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bc766a-6fa5-4e70-a4ff-4519e55dcf9b service nova] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 02:59:59.438595 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bc766a-6fa5-4e70-a4ff-4519e55dcf9b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 02:59:59.440158 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-17bc766a-6fa5-4e70-a4ff-4519e55dcf9b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 02:59:59.536821 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bc766a-6fa5-4e70-a4ff-4519e55dcf9b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 02:59:59.538878 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bc766a-6fa5-4e70-a4ff-4519e55dcf9b service nova] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 02:59:59.558786 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 511/1037] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 02:59:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:00:01.943205 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4491c86f-258d-41b9-8ec3-a411e52ad9a3 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:00:01.943758 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4491c86f-258d-41b9-8ec3-a411e52ad9a3 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:00:01.943758 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4491c86f-258d-41b9-8ec3-a411e52ad9a3 None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:00:01.945638 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 527/1038] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 03:00:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 03:00:01.950960 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5142ece-86ba-4193-81ee-1f7e3cc3da85 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:00:01.951385 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5142ece-86ba-4193-81ee-1f7e3cc3da85 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:00:01.951515 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5142ece-86ba-4193-81ee-1f7e3cc3da85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:00:02.022985 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-d5142ece-86ba-4193-81ee-1f7e3cc3da85 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:00:02.024232 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-d5142ece-86ba-4193-81ee-1f7e3cc3da85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:00:02.068333 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:00:02.133019 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 512/1039] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 03:00:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:00:02.425161 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4491c86f-258d-41b9-8ec3-a411e52ad9a3 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:00:02.426715 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-4491c86f-258d-41b9-8ec3-a411e52ad9a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:00:02.502503 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:00:02.596097 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4491c86f-258d-41b9-8ec3-a411e52ad9a3 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:00:02.736257 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0823df93-1e6b-4679-969d-9f80739270a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b07bf0faca47b28b50990d1720853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:00:02.736864 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0823df93-1e6b-4679-969d-9f80739270a7 service nova] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:00:02.737146 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0823df93-1e6b-4679-969d-9f80739270a7 service nova] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:00:02.737420 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0823df93-1e6b-4679-969d-9f80739270a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:00:02.739030 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-0823df93-1e6b-4679-969d-9f80739270a7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:00:02.764808 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:00:02.838176 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0823df93-1e6b-4679-969d-9f80739270a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:00:02.839787 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0823df93-1e6b-4679-969d-9f80739270a7 service nova] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:00:02.859120 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 528/1040] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:00:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:00:05.470095 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ec0803-c120-4567-aaac-70884a397b58 None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:00:05.470695 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ec0803-c120-4567-aaac-70884a397b58 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:00:05.470695 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ec0803-c120-4567-aaac-70884a397b58 None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:00:05.472005 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 513/1041] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 03:00:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 03:00:05.477443 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894be400-089a-45c5-b3e1-11c7ca11af00 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:00:05.477906 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894be400-089a-45c5-b3e1-11c7ca11af00 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:00:05.478063 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894be400-089a-45c5-b3e1-11c7ca11af00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:00:05.546029 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-894be400-089a-45c5-b3e1-11c7ca11af00 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:00:05.547242 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-894be400-089a-45c5-b3e1-11c7ca11af00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:00:05.647193 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:00:05.650103 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 529/1042] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 03:00:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:00:05.942589 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f6e631-2d5b-4884-a93a-1035544d226c None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:00:05.942900 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f6e631-2d5b-4884-a93a-1035544d226c None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:00:05.943209 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f6e631-2d5b-4884-a93a-1035544d226c None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:00:05.944693 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 514/1043] 10.222.0.39 () {54 vars in 961 bytes} [Fri Oct 14 03:00:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 03:00:05.949983 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2167b92a-b75c-4948-9ba6-bf2da9c01c95 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:00:05.950276 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2167b92a-b75c-4948-9ba6-bf2da9c01c95 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:00:05.950560 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2167b92a-b75c-4948-9ba6-bf2da9c01c95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:00:06.034059 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-2167b92a-b75c-4948-9ba6-bf2da9c01c95 None None] MFA Rules not processed for user `a6b07bf0faca47b28b50990d1720853f`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:00:06.035283 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-2167b92a-b75c-4948-9ba6-bf2da9c01c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:00:06.156064 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 530/1044] 10.222.0.39 () {58 vars in 1058 bytes} [Fri Oct 14 03:00:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:00:06.164456 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f6e631-2d5b-4884-a93a-1035544d226c None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:00:06.165998 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-c7f6e631-2d5b-4884-a93a-1035544d226c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:00:06.319596 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f6e631-2d5b-4884-a93a-1035544d226c None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:00:06.341309 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:00:06.439509 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b01653-3018-4455-b7e6-be3c58a5cf31 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b07bf0faca47b28b50990d1720853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:00:06.440120 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b01653-3018-4455-b7e6-be3c58a5cf31 service nova] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:00:06.440399 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b01653-3018-4455-b7e6-be3c58a5cf31 service nova] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:00:06.440668 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b01653-3018-4455-b7e6-be3c58a5cf31 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:00:06.442278 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-d8b01653-3018-4455-b7e6-be3c58a5cf31 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:00:06.542431 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b01653-3018-4455-b7e6-be3c58a5cf31 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:00:06.544019 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b01653-3018-4455-b7e6-be3c58a5cf31 service nova] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:00:06.562879 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 515/1045] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:00:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:00:06.587127 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66447036-dadd-4c2b-85fe-4db8bf14ba0c None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:00:06.587578 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66447036-dadd-4c2b-85fe-4db8bf14ba0c None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:00:06.587677 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66447036-dadd-4c2b-85fe-4db8bf14ba0c None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:00:06.589160 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 531/1046] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 03:00:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 03:00:06.595155 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3b8283-326d-49c4-9dd3-154e43dd3589 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:00:06.595446 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3b8283-326d-49c4-9dd3-154e43dd3589 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:00:06.595748 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3b8283-326d-49c4-9dd3-154e43dd3589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:00:06.629282 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:00:06.664401 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-9a3b8283-326d-49c4-9dd3-154e43dd3589 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:00:06.665628 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-9a3b8283-326d-49c4-9dd3-154e43dd3589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:00:06.772715 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 516/1047] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 03:00:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:00:07.104577 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4851c283-0a1e-46b8-a266-991d3e8fcb0e None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:00:07.104577 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4851c283-0a1e-46b8-a266-991d3e8fcb0e None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:00:07.104577 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4851c283-0a1e-46b8-a266-991d3e8fcb0e None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:00:07.104577 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 532/1048] 10.222.0.39 () {54 vars in 963 bytes} [Fri Oct 14 03:00:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 03:00:07.110882 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe7b372-ef7b-4bab-93a6-36f1bdd9e6b3 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:00:07.110882 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe7b372-ef7b-4bab-93a6-36f1bdd9e6b3 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:00:07.111927 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe7b372-ef7b-4bab-93a6-36f1bdd9e6b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:00:07.192094 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-abe7b372-ef7b-4bab-93a6-36f1bdd9e6b3 None None] MFA Rules not processed for user `ab599d98b0624a2e89090312bcab713f`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:00:07.193389 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-abe7b372-ef7b-4bab-93a6-36f1bdd9e6b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:00:07.243592 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:00:07.276117 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 517/1049] 10.222.0.39 () {58 vars in 1060 bytes} [Fri Oct 14 03:00:07 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:00:07.284473 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4851c283-0a1e-46b8-a266-991d3e8fcb0e None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:00:07.286116 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-4851c283-0a1e-46b8-a266-991d3e8fcb0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:00:07.342394 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:00:07.399955 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4851c283-0a1e-46b8-a266-991d3e8fcb0e None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:00:07.480442 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a4def5-3730-4d21-98ba-64599777322b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab599d98b0624a2e89090312bcab713f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:00:07.481329 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a4def5-3730-4d21-98ba-64599777322b service cinder] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:00:07.481712 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a4def5-3730-4d21-98ba-64599777322b service cinder] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:00:07.482111 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a4def5-3730-4d21-98ba-64599777322b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:00:07.484096 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-30a4def5-3730-4d21-98ba-64599777322b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:00:07.590354 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:00:07.603628 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a4def5-3730-4d21-98ba-64599777322b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:00:07.605057 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a4def5-3730-4d21-98ba-64599777322b service cinder] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:00:07.623261 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 533/1050] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:00:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:00:08.926945 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691bffd7-a956-4104-9cd6-647e88d7280c None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:00:08.927513 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691bffd7-a956-4104-9cd6-647e88d7280c None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:00:08.927513 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691bffd7-a956-4104-9cd6-647e88d7280c None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:00:08.928843 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 518/1051] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 03:00:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 14 03:00:08.934608 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd6992a-6d7a-4c21-966a-99a9d7e1d386 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:00:08.935041 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd6992a-6d7a-4c21-966a-99a9d7e1d386 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:00:08.935164 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd6992a-6d7a-4c21-966a-99a9d7e1d386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:00:09.001412 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-6dd6992a-6d7a-4c21-966a-99a9d7e1d386 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:00:09.002662 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-6dd6992a-6d7a-4c21-966a-99a9d7e1d386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:00:09.104951 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 534/1052] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 03:00:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:00:09.393566 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691bffd7-a956-4104-9cd6-647e88d7280c None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:00:09.395254 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-691bffd7-a956-4104-9cd6-647e88d7280c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:00:09.548286 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691bffd7-a956-4104-9cd6-647e88d7280c None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:00:09.605700 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:00:09.674563 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd0ff81-7cf9-40e4-bb3d-c4eb49967479 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b07bf0faca47b28b50990d1720853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:00:09.675175 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd0ff81-7cf9-40e4-bb3d-c4eb49967479 service nova] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:00:09.675455 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd0ff81-7cf9-40e4-bb3d-c4eb49967479 service nova] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:00:09.675725 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd0ff81-7cf9-40e4-bb3d-c4eb49967479 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:00:09.678658 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-0bd0ff81-7cf9-40e4-bb3d-c4eb49967479 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:00:09.773880 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd0ff81-7cf9-40e4-bb3d-c4eb49967479 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:00:09.775501 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd0ff81-7cf9-40e4-bb3d-c4eb49967479 service nova] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:00:09.794773 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 519/1053] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:00:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:00:12.104446 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290f4bab-9995-4c6f-9060-34edbef20d13 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:00:12.104981 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290f4bab-9995-4c6f-9060-34edbef20d13 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:00:12.104981 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290f4bab-9995-4c6f-9060-34edbef20d13 None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:00:12.106366 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 535/1054] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 03:00:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 03:00:12.111871 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c9a4b8-93d2-49dc-9b24-0622f55822d9 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:00:12.112258 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c9a4b8-93d2-49dc-9b24-0622f55822d9 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:00:12.112436 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c9a4b8-93d2-49dc-9b24-0622f55822d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:00:12.178167 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-d9c9a4b8-93d2-49dc-9b24-0622f55822d9 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:00:12.179906 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-d9c9a4b8-93d2-49dc-9b24-0622f55822d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:00:12.187135 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:00:12.280476 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 520/1055] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 03:00:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:00:12.570375 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290f4bab-9995-4c6f-9060-34edbef20d13 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:00:12.571882 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-290f4bab-9995-4c6f-9060-34edbef20d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:00:12.635065 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:00:12.721534 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290f4bab-9995-4c6f-9060-34edbef20d13 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:00:12.841446 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dad31a4-c4c7-4d36-abd7-464db1fdff7a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b07bf0faca47b28b50990d1720853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:00:12.842087 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dad31a4-c4c7-4d36-abd7-464db1fdff7a service nova] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:00:12.842368 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dad31a4-c4c7-4d36-abd7-464db1fdff7a service nova] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:00:12.842641 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dad31a4-c4c7-4d36-abd7-464db1fdff7a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:00:12.844232 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-1dad31a4-c4c7-4d36-abd7-464db1fdff7a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:00:12.864826 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:00:12.940477 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dad31a4-c4c7-4d36-abd7-464db1fdff7a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:00:12.942190 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dad31a4-c4c7-4d36-abd7-464db1fdff7a service nova] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:00:12.961631 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 536/1056] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:00:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:00:15.296400 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1893639-fd9e-4330-9192-aa8f2fd619b1 None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:00:15.296885 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1893639-fd9e-4330-9192-aa8f2fd619b1 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:00:15.297320 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1893639-fd9e-4330-9192-aa8f2fd619b1 None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:00:15.298422 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 521/1057] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 03:00:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 03:00:15.304420 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa05c4b7-436e-4ec0-b6ff-d20386f68d31 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:00:15.304855 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa05c4b7-436e-4ec0-b6ff-d20386f68d31 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:00:15.305072 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa05c4b7-436e-4ec0-b6ff-d20386f68d31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:00:15.377661 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-fa05c4b7-436e-4ec0-b6ff-d20386f68d31 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:00:15.379024 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-fa05c4b7-436e-4ec0-b6ff-d20386f68d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:00:15.473373 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:00:15.479033 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 537/1058] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 03:00:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:00:15.773994 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1893639-fd9e-4330-9192-aa8f2fd619b1 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:00:15.775656 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-e1893639-fd9e-4330-9192-aa8f2fd619b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:00:15.914612 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:00:15.925187 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1893639-fd9e-4330-9192-aa8f2fd619b1 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:00:16.053549 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eece5075-58cb-4bf6-9093-f6503c592a54 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b07bf0faca47b28b50990d1720853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:00:16.054131 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eece5075-58cb-4bf6-9093-f6503c592a54 service nova] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:00:16.054452 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eece5075-58cb-4bf6-9093-f6503c592a54 service nova] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:00:16.054696 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eece5075-58cb-4bf6-9093-f6503c592a54 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:00:16.056391 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-eece5075-58cb-4bf6-9093-f6503c592a54 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:00:16.155372 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eece5075-58cb-4bf6-9093-f6503c592a54 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:00:16.157032 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eece5075-58cb-4bf6-9093-f6503c592a54 service nova] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:00:16.167817 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:00:16.176747 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 522/1059] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:00:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:00:19.110394 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769c64e8-9e5a-4fc5-ad42-894e18d13a51 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:00:19.110934 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769c64e8-9e5a-4fc5-ad42-894e18d13a51 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:00:19.111104 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769c64e8-9e5a-4fc5-ad42-894e18d13a51 None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:00:19.112874 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 538/1060] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 03:00:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 03:00:19.119551 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92c7fcd-c777-4980-97c3-b763acb4594b None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:00:19.119983 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92c7fcd-c777-4980-97c3-b763acb4594b None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:00:19.120337 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92c7fcd-c777-4980-97c3-b763acb4594b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:00:19.201252 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-b92c7fcd-c777-4980-97c3-b763acb4594b None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:00:19.202497 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-b92c7fcd-c777-4980-97c3-b763acb4594b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:00:19.298911 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 523/1061] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 03:00:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:00:19.583778 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769c64e8-9e5a-4fc5-ad42-894e18d13a51 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:00:19.585924 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-769c64e8-9e5a-4fc5-ad42-894e18d13a51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:00:19.742525 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769c64e8-9e5a-4fc5-ad42-894e18d13a51 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:00:19.813612 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:00:19.875161 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3178eb2b-098f-4ded-a784-82767d032888 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b07bf0faca47b28b50990d1720853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:00:19.875778 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3178eb2b-098f-4ded-a784-82767d032888 service nova] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:00:19.876093 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3178eb2b-098f-4ded-a784-82767d032888 service nova] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:00:19.876384 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3178eb2b-098f-4ded-a784-82767d032888 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:00:19.878156 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-3178eb2b-098f-4ded-a784-82767d032888 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:00:19.972326 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3178eb2b-098f-4ded-a784-82767d032888 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:00:19.974048 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3178eb2b-098f-4ded-a784-82767d032888 service nova] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:00:19.994051 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 539/1062] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:00:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:00:22.232247 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad965c-f8bc-4904-85d2-b2cc7cdae62e None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:00:22.232815 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad965c-f8bc-4904-85d2-b2cc7cdae62e None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:00:22.232815 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad965c-f8bc-4904-85d2-b2cc7cdae62e None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:00:22.234420 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 524/1063] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 03:00:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 03:00:22.239867 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a10c538-dbd0-47cb-8b22-839d9c44cfc4 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:00:22.240269 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a10c538-dbd0-47cb-8b22-839d9c44cfc4 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:00:22.240483 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a10c538-dbd0-47cb-8b22-839d9c44cfc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:00:22.305942 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-4a10c538-dbd0-47cb-8b22-839d9c44cfc4 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:00:22.307051 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-4a10c538-dbd0-47cb-8b22-839d9c44cfc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:00:22.338853 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:00:22.402011 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 540/1064] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 03:00:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:00:22.692024 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ad965c-f8bc-4904-85d2-b2cc7cdae62e None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:00:22.693630 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-a7ad965c-f8bc-4904-85d2-b2cc7cdae62e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:00:22.770044 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:00:22.838759 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ad965c-f8bc-4904-85d2-b2cc7cdae62e None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:00:22.957713 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcb3da3-0c4f-49f6-812a-45d1ee034d57 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b07bf0faca47b28b50990d1720853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:00:22.958269 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcb3da3-0c4f-49f6-812a-45d1ee034d57 service nova] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:00:22.958557 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcb3da3-0c4f-49f6-812a-45d1ee034d57 service nova] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:00:22.958832 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcb3da3-0c4f-49f6-812a-45d1ee034d57 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:00:22.960938 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-0dcb3da3-0c4f-49f6-812a-45d1ee034d57 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:00:23.000268 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:00:23.043934 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba78f161-6732-4705-adeb-96e9db246850 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:00:23.044215 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba78f161-6732-4705-adeb-96e9db246850 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:00:23.044489 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba78f161-6732-4705-adeb-96e9db246850 None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:00:23.045854 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 541/1065] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 03:00:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 14 03:00:23.050734 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f24d7-be83-4dda-bd81-0f0d15c15960 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:00:23.051544 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f24d7-be83-4dda-bd81-0f0d15c15960 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:00:23.051544 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f24d7-be83-4dda-bd81-0f0d15c15960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:00:23.053670 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcb3da3-0c4f-49f6-812a-45d1ee034d57 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:00:23.055299 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcb3da3-0c4f-49f6-812a-45d1ee034d57 service nova] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:00:23.075292 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 525/1066] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:00:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:00:23.122660 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-c79f24d7-be83-4dda-bd81-0f0d15c15960 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:00:23.123928 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-c79f24d7-be83-4dda-bd81-0f0d15c15960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:00:23.222363 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 542/1067] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 03:00:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 03:00:23.525129 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bf0c96-1215-4bf1-ba94-b4aa1df59aac None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:00:23.525595 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bf0c96-1215-4bf1-ba94-b4aa1df59aac None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:00:23.525711 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bf0c96-1215-4bf1-ba94-b4aa1df59aac None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:00:23.527039 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 526/1068] 10.222.0.39 () {54 vars in 962 bytes} [Fri Oct 14 03:00:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 03:00:23.532414 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22776226-d211-4d45-8b23-e19cc8030570 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:00:23.532812 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22776226-d211-4d45-8b23-e19cc8030570 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:00:23.532994 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22776226-d211-4d45-8b23-e19cc8030570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:00:23.549429 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:00:23.611479 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-22776226-d211-4d45-8b23-e19cc8030570 None None] MFA Rules not processed for user `6a3cd1fbe6df48f8aa90d4ea69905e9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:00:23.612672 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-22776226-d211-4d45-8b23-e19cc8030570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:00:23.689792 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 543/1069] 10.222.0.39 () {58 vars in 1059 bytes} [Fri Oct 14 03:00:23 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:00:23.699577 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bf0c96-1215-4bf1-ba94-b4aa1df59aac None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:00:23.701185 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-08bf0c96-1215-4bf1-ba94-b4aa1df59aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:00:23.804103 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bf0c96-1215-4bf1-ba94-b4aa1df59aac None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:00:23.863339 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:00:23.880004 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c95bcb-721f-4f48-a596-08be08e07e25 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a3cd1fbe6df48f8aa90d4ea69905e9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:00:23.880615 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c95bcb-721f-4f48-a596-08be08e07e25 service glance] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:00:23.880895 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c95bcb-721f-4f48-a596-08be08e07e25 service glance] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:00:23.881171 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c95bcb-721f-4f48-a596-08be08e07e25 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:00:23.882790 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-a2c95bcb-721f-4f48-a596-08be08e07e25 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:00:23.993544 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c95bcb-721f-4f48-a596-08be08e07e25 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:00:23.995041 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c95bcb-721f-4f48-a596-08be08e07e25 service glance] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:00:24.018380 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 527/1070] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:00:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:00:24.935486 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7769526f-65f5-4ebc-bca8-3d8565996d93 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:00:24.935971 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7769526f-65f5-4ebc-bca8-3d8565996d93 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:00:24.936083 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7769526f-65f5-4ebc-bca8-3d8565996d93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:00:25.004535 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-7769526f-65f5-4ebc-bca8-3d8565996d93 None None] MFA Rules not processed for user `44b37050076b42e6af1a67bd7d6d06e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:00:25.009480 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-7769526f-65f5-4ebc-bca8-3d8565996d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:00:25.060997 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:00:25.092346 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 544/1071] 10.222.0.39 () {58 vars in 1009 bytes} [Fri Oct 14 03:00:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 03:00:25.139279 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a96c365-572a-4fae-a5c7-051bc2a22474 None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:00:25.139749 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a96c365-572a-4fae-a5c7-051bc2a22474 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:00:25.139955 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a96c365-572a-4fae-a5c7-051bc2a22474 None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:00:25.141468 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 528/1072] 10.222.0.39 () {54 vars in 943 bytes} [Fri Oct 14 03:00:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 03:00:25.148199 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e640abe-3bc2-469f-9106-7f4a2710ac09 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:00:25.148484 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e640abe-3bc2-469f-9106-7f4a2710ac09 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:00:25.148758 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e640abe-3bc2-469f-9106-7f4a2710ac09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:00:25.224437 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-7e640abe-3bc2-469f-9106-7f4a2710ac09 None None] MFA Rules not processed for user `5c5e07da64804243bc18862072b027a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:00:25.225652 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-7e640abe-3bc2-469f-9106-7f4a2710ac09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:00:25.338237 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 545/1073] 10.222.0.39 () {58 vars in 1040 bytes} [Fri Oct 14 03:00:25 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:00:25.347837 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a96c365-572a-4fae-a5c7-051bc2a22474 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:00:25.349326 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-7a96c365-572a-4fae-a5c7-051bc2a22474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:00:25.358881 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4117288-f498-4ceb-9497-b1e46524231f None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:00:25.359178 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4117288-f498-4ceb-9497-b1e46524231f None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:00:25.359456 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4117288-f498-4ceb-9497-b1e46524231f None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:00:25.360800 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 546/1074] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 03:00:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 14 03:00:25.366023 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55e91b7-e2b1-432f-a7d8-b581cfcc3cf6 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:00:25.366308 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55e91b7-e2b1-432f-a7d8-b581cfcc3cf6 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:00:25.366580 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55e91b7-e2b1-432f-a7d8-b581cfcc3cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:00:25.402896 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:00:25.433684 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-d55e91b7-e2b1-432f-a7d8-b581cfcc3cf6 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:00:25.435003 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-d55e91b7-e2b1-432f-a7d8-b581cfcc3cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:00:25.439037 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:00:25.487573 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a96c365-572a-4fae-a5c7-051bc2a22474 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:00:25.536106 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 547/1075] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 03:00:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 03:00:25.592191 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566812a6-0c3f-4213-a3ab-5f580ce8cc53 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5e07da64804243bc18862072b027a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:00:25.592730 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566812a6-0c3f-4213-a3ab-5f580ce8cc53 service swift] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:00:25.593013 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566812a6-0c3f-4213-a3ab-5f580ce8cc53 service swift] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:00:25.593288 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566812a6-0c3f-4213-a3ab-5f580ce8cc53 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:00:25.594965 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-566812a6-0c3f-4213-a3ab-5f580ce8cc53 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:00:25.665608 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:00:25.677441 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566812a6-0c3f-4213-a3ab-5f580ce8cc53 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:00:25.679700 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566812a6-0c3f-4213-a3ab-5f580ce8cc53 service swift] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:00:25.687799 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 529/1076] 10.222.0.39 () {60 vars in 1384 bytes} [Fri Oct 14 03:00:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 14 03:00:25.821695 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55e91b7-e2b1-432f-a7d8-b581cfcc3cf6 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:00:25.822940 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-d55e91b7-e2b1-432f-a7d8-b581cfcc3cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:00:25.933018 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:00:25.937031 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55e91b7-e2b1-432f-a7d8-b581cfcc3cf6 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:00:26.050888 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acc3c60-3964-4a75-9b35-3d5376eaa0fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b07bf0faca47b28b50990d1720853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:00:26.051500 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acc3c60-3964-4a75-9b35-3d5376eaa0fd service nova] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:00:26.051782 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acc3c60-3964-4a75-9b35-3d5376eaa0fd service nova] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:00:26.052056 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acc3c60-3964-4a75-9b35-3d5376eaa0fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:00:26.053706 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-3acc3c60-3964-4a75-9b35-3d5376eaa0fd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:00:26.157308 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acc3c60-3964-4a75-9b35-3d5376eaa0fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:00:26.158998 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acc3c60-3964-4a75-9b35-3d5376eaa0fd service nova] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:00:26.177654 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 548/1077] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:00:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:00:28.478385 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d3869a-a478-473f-96f4-af7c453cde4e None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:00:28.479158 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d3869a-a478-473f-96f4-af7c453cde4e None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:00:28.479158 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d3869a-a478-473f-96f4-af7c453cde4e None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:00:28.480299 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 530/1078] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 03:00:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 03:00:28.487087 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3305d471-c2f5-4b9b-a3d8-0805d40566be None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:00:28.487623 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3305d471-c2f5-4b9b-a3d8-0805d40566be None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:00:28.487623 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3305d471-c2f5-4b9b-a3d8-0805d40566be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:00:28.493309 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:00:28.572990 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-3305d471-c2f5-4b9b-a3d8-0805d40566be None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:00:28.574620 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-3305d471-c2f5-4b9b-a3d8-0805d40566be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:00:28.697805 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 549/1079] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 03:00:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:00:28.987652 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d3869a-a478-473f-96f4-af7c453cde4e None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:00:28.989170 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-06d3869a-a478-473f-96f4-af7c453cde4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:00:29.154940 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d3869a-a478-473f-96f4-af7c453cde4e None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:00:29.226339 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:00:29.281041 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824ad264-e410-4b6a-8898-93a601b707f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b07bf0faca47b28b50990d1720853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:00:29.283844 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ad264-e410-4b6a-8898-93a601b707f3 service nova] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:00:29.283844 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ad264-e410-4b6a-8898-93a601b707f3 service nova] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:00:29.283844 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ad264-e410-4b6a-8898-93a601b707f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:00:29.284173 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-824ad264-e410-4b6a-8898-93a601b707f3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:00:29.376649 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824ad264-e410-4b6a-8898-93a601b707f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:00:29.378661 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824ad264-e410-4b6a-8898-93a601b707f3 service nova] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:00:29.401628 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 531/1080] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:00:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:00:31.727484 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b581d5-0c13-4295-a733-39a46d5b13d1 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:00:31.728039 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b581d5-0c13-4295-a733-39a46d5b13d1 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:00:31.728039 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b581d5-0c13-4295-a733-39a46d5b13d1 None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:00:31.729515 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 550/1081] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 03:00:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 03:00:31.734796 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e5bb79-4581-429e-8c0a-8ee789628fa4 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:00:31.735229 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e5bb79-4581-429e-8c0a-8ee789628fa4 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:00:31.735318 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e5bb79-4581-429e-8c0a-8ee789628fa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:00:31.811320 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-29e5bb79-4581-429e-8c0a-8ee789628fa4 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:00:31.812442 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-29e5bb79-4581-429e-8c0a-8ee789628fa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:00:31.840718 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:00:31.921742 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 532/1082] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 03:00:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:00:32.211177 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b581d5-0c13-4295-a733-39a46d5b13d1 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:00:32.212756 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-22b581d5-0c13-4295-a733-39a46d5b13d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:00:32.302209 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:00:32.373178 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b581d5-0c13-4295-a733-39a46d5b13d1 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:00:32.492954 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1b5469-d529-4c2d-a0ca-fddc7413a39f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b07bf0faca47b28b50990d1720853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:00:32.493536 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1b5469-d529-4c2d-a0ca-fddc7413a39f service nova] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:00:32.493838 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1b5469-d529-4c2d-a0ca-fddc7413a39f service nova] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:00:32.494107 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1b5469-d529-4c2d-a0ca-fddc7413a39f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:00:32.495706 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-fa1b5469-d529-4c2d-a0ca-fddc7413a39f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:00:32.541492 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:00:32.596143 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1b5469-d529-4c2d-a0ca-fddc7413a39f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:00:32.597745 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1b5469-d529-4c2d-a0ca-fddc7413a39f service nova] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:00:32.617218 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 551/1083] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:00:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:00:35.010195 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15845958-2a7c-44f5-bd20-192ddf9dae6e None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:00:35.011367 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15845958-2a7c-44f5-bd20-192ddf9dae6e None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:00:35.012120 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15845958-2a7c-44f5-bd20-192ddf9dae6e None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:00:35.014161 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 533/1084] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 03:00:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 03:00:35.019906 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad101707-2b8c-4e71-a7d1-3297e6cbb0a6 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:00:35.020608 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad101707-2b8c-4e71-a7d1-3297e6cbb0a6 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:00:35.020608 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad101707-2b8c-4e71-a7d1-3297e6cbb0a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:00:35.091502 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-ad101707-2b8c-4e71-a7d1-3297e6cbb0a6 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:00:35.092903 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-ad101707-2b8c-4e71-a7d1-3297e6cbb0a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:00:35.205742 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 552/1085] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 03:00:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:00:35.500212 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15845958-2a7c-44f5-bd20-192ddf9dae6e None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:00:35.501741 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-15845958-2a7c-44f5-bd20-192ddf9dae6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:00:35.663692 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15845958-2a7c-44f5-bd20-192ddf9dae6e None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:00:35.673097 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:00:35.789352 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87859d29-8700-44db-b633-075eca6e80e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b07bf0faca47b28b50990d1720853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:00:35.789894 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87859d29-8700-44db-b633-075eca6e80e6 service nova] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:00:35.790282 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87859d29-8700-44db-b633-075eca6e80e6 service nova] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:00:35.790562 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87859d29-8700-44db-b633-075eca6e80e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:00:35.792158 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-87859d29-8700-44db-b633-075eca6e80e6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:00:35.891881 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87859d29-8700-44db-b633-075eca6e80e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:00:35.893546 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87859d29-8700-44db-b633-075eca6e80e6 service nova] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:00:35.913424 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 534/1086] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:00:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:00:45.205971 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=91 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:00:45.910101 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=94 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:00:49.650815 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21719f87-22d0-4609-b33d-25121fa77c2e None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:00:49.651355 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21719f87-22d0-4609-b33d-25121fa77c2e None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:00:49.651355 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21719f87-22d0-4609-b33d-25121fa77c2e None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:00:49.652709 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 553/1087] 10.222.0.39 () {54 vars in 918 bytes} [Fri Oct 14 03:00:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 03:00:49.657745 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b8cae-6a09-47f0-a263-d414066deead None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:00:49.658184 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b8cae-6a09-47f0-a263-d414066deead None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:00:49.658355 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b8cae-6a09-47f0-a263-d414066deead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:00:49.731728 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-261b8cae-6a09-47f0-a263-d414066deead None None] MFA Rules not processed for user `922ef7811aed450baff7f936011b0af1`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:00:49.732929 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-261b8cae-6a09-47f0-a263-d414066deead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:00:49.837047 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 535/1088] 10.222.0.39 () {58 vars in 1015 bytes} [Fri Oct 14 03:00:49 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:00:49.850878 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19fd136-7b89-48bb-b841-701842dc1c76 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:00:49.851162 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19fd136-7b89-48bb-b841-701842dc1c76 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:00:49.851436 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19fd136-7b89-48bb-b841-701842dc1c76 None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:00:49.852756 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 554/1089] 10.222.0.39 () {54 vars in 918 bytes} [Fri Oct 14 03:00:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 03:00:49.857346 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932f4233-a0cf-4e8e-93e4-18b7b00bc892 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:00:49.857636 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932f4233-a0cf-4e8e-93e4-18b7b00bc892 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:00:49.857951 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932f4233-a0cf-4e8e-93e4-18b7b00bc892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:00:49.933903 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:00:49.936860 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-932f4233-a0cf-4e8e-93e4-18b7b00bc892 None None] MFA Rules not processed for user `a6b07bf0faca47b28b50990d1720853f`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:00:49.938085 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-932f4233-a0cf-4e8e-93e4-18b7b00bc892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:00:50.053945 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 536/1090] 10.222.0.39 () {58 vars in 1015 bytes} [Fri Oct 14 03:00:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:00:52.606554 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d41481-b940-4bd4-aab3-2838ee51d926 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:00:52.607090 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d41481-b940-4bd4-aab3-2838ee51d926 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:00:52.607186 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d41481-b940-4bd4-aab3-2838ee51d926 None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:00:52.608450 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 555/1091] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 14 03:00:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 03:00:52.614052 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f6e39b-7600-4f5a-8954-e37a3e109c19 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:00:52.614451 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f6e39b-7600-4f5a-8954-e37a3e109c19 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:00:52.614607 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f6e39b-7600-4f5a-8954-e37a3e109c19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:00:52.680049 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-94f6e39b-7600-4f5a-8954-e37a3e109c19 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:00:52.681253 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-94f6e39b-7600-4f5a-8954-e37a3e109c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:00:52.734750 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:00:52.776453 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 537/1092] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 14 03:00:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 03:00:53.063546 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d41481-b940-4bd4-aab3-2838ee51d926 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:00:53.065065 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-90d41481-b940-4bd4-aab3-2838ee51d926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:00:53.208497 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d41481-b940-4bd4-aab3-2838ee51d926 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:00:53.279947 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:00:53.320083 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c801c679-6d27-42f5-84b5-fb4fcec00daa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b07bf0faca47b28b50990d1720853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:00:53.320687 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c801c679-6d27-42f5-84b5-fb4fcec00daa service nova] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:00:53.320968 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c801c679-6d27-42f5-84b5-fb4fcec00daa service nova] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:00:53.321242 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c801c679-6d27-42f5-84b5-fb4fcec00daa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:00:53.322876 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-c801c679-6d27-42f5-84b5-fb4fcec00daa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:00:53.409405 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c801c679-6d27-42f5-84b5-fb4fcec00daa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:00:53.411017 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c801c679-6d27-42f5-84b5-fb4fcec00daa service nova] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:00:53.429416 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 556/1093] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:00:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:00:54.566450 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18105262-8257-40e6-abcf-b951707dfc22 None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:00:54.566987 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18105262-8257-40e6-abcf-b951707dfc22 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:00:54.566987 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18105262-8257-40e6-abcf-b951707dfc22 None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:00:54.568981 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 538/1094] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 14 03:00:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 03:00:54.575147 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efd1e05-d12a-40c7-8ed0-e5f64e2296da None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:00:54.575544 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efd1e05-d12a-40c7-8ed0-e5f64e2296da None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:00:54.575687 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efd1e05-d12a-40c7-8ed0-e5f64e2296da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:00:54.651921 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-1efd1e05-d12a-40c7-8ed0-e5f64e2296da None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:00:54.653119 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-1efd1e05-d12a-40c7-8ed0-e5f64e2296da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:00:54.680454 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:00:54.749425 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 557/1095] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 14 03:00:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:00:54.899875 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18105262-8257-40e6-abcf-b951707dfc22 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:00:54.901409 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-18105262-8257-40e6-abcf-b951707dfc22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:00:55.044051 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18105262-8257-40e6-abcf-b951707dfc22 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:00:55.083481 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:00:55.155626 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef982e76-ad5c-4008-83c8-2f3b280b3995 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b07bf0faca47b28b50990d1720853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:00:55.156156 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef982e76-ad5c-4008-83c8-2f3b280b3995 service nova] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:00:55.156437 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef982e76-ad5c-4008-83c8-2f3b280b3995 service nova] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:00:55.156710 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef982e76-ad5c-4008-83c8-2f3b280b3995 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:00:55.158367 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-ef982e76-ad5c-4008-83c8-2f3b280b3995 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:00:55.250743 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef982e76-ad5c-4008-83c8-2f3b280b3995 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:00:55.252328 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef982e76-ad5c-4008-83c8-2f3b280b3995 service nova] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:00:55.269728 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 539/1096] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:00:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:00:56.391714 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047ade57-c709-47e3-85ad-f7cde24bdb46 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:00:56.392148 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047ade57-c709-47e3-85ad-f7cde24bdb46 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:00:56.392297 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047ade57-c709-47e3-85ad-f7cde24bdb46 None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:00:56.393690 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 558/1097] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 14 03:00:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 03:00:56.400258 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e943210c-dccf-4727-be38-b80c3e26cc90 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:00:56.400665 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e943210c-dccf-4727-be38-b80c3e26cc90 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:00:56.400807 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e943210c-dccf-4727-be38-b80c3e26cc90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:00:56.452874 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:00:56.474153 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-e943210c-dccf-4727-be38-b80c3e26cc90 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:00:56.475351 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-e943210c-dccf-4727-be38-b80c3e26cc90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:00:56.569129 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 540/1098] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 14 03:00:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:00:56.720781 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047ade57-c709-47e3-85ad-f7cde24bdb46 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:00:56.722317 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-047ade57-c709-47e3-85ad-f7cde24bdb46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:00:56.865953 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047ade57-c709-47e3-85ad-f7cde24bdb46 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:00:56.877740 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:00:56.978473 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537501aa-b085-48dd-a28f-fa815a8cd5c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b07bf0faca47b28b50990d1720853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:00:56.979029 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537501aa-b085-48dd-a28f-fa815a8cd5c5 service nova] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:00:56.979314 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537501aa-b085-48dd-a28f-fa815a8cd5c5 service nova] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:00:56.979594 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537501aa-b085-48dd-a28f-fa815a8cd5c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:00:56.981197 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-537501aa-b085-48dd-a28f-fa815a8cd5c5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:00:57.071233 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537501aa-b085-48dd-a28f-fa815a8cd5c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:00:57.072813 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537501aa-b085-48dd-a28f-fa815a8cd5c5 service nova] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:00:57.091313 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 559/1099] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:00:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:00:58.188941 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5448aae4-94be-4ee4-aea8-b694077c612b None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:00:58.189478 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5448aae4-94be-4ee4-aea8-b694077c612b None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:00:58.189604 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5448aae4-94be-4ee4-aea8-b694077c612b None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:00:58.190906 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 541/1100] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 14 03:00:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 03:00:58.197244 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fbc32e-3562-4430-a50f-c07546a67b9d None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:00:58.197665 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fbc32e-3562-4430-a50f-c07546a67b9d None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:00:58.197826 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fbc32e-3562-4430-a50f-c07546a67b9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:00:58.217484 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:00:58.274981 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-d2fbc32e-3562-4430-a50f-c07546a67b9d None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:00:58.276104 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-d2fbc32e-3562-4430-a50f-c07546a67b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:00:58.370100 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 560/1101] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 14 03:00:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:00:58.522095 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5448aae4-94be-4ee4-aea8-b694077c612b None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:00:58.524166 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-5448aae4-94be-4ee4-aea8-b694077c612b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:00:58.631327 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:00:58.666555 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5448aae4-94be-4ee4-aea8-b694077c612b None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:00:58.775617 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eea4bb-0c00-44fd-a2a5-b3dd70613f3d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b07bf0faca47b28b50990d1720853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:00:58.776219 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eea4bb-0c00-44fd-a2a5-b3dd70613f3d service nova] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:00:58.776499 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eea4bb-0c00-44fd-a2a5-b3dd70613f3d service nova] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:00:58.776768 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eea4bb-0c00-44fd-a2a5-b3dd70613f3d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:00:58.778376 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-44eea4bb-0c00-44fd-a2a5-b3dd70613f3d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:00:58.854803 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:00:58.868514 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eea4bb-0c00-44fd-a2a5-b3dd70613f3d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:00:58.870127 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eea4bb-0c00-44fd-a2a5-b3dd70613f3d service nova] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:00:58.888045 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 542/1102] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:00:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:01:00.001015 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097eae33-3877-4777-8464-5360ba5f0390 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:01:00.001588 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097eae33-3877-4777-8464-5360ba5f0390 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:01:00.001588 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097eae33-3877-4777-8464-5360ba5f0390 None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:01:00.003003 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 561/1103] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 14 03:00:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 03:01:00.010051 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6415d92f-8bb5-4b26-a2d4-8716e32a2349 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:01:00.010516 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6415d92f-8bb5-4b26-a2d4-8716e32a2349 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:01:00.010607 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6415d92f-8bb5-4b26-a2d4-8716e32a2349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:01:00.085919 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-6415d92f-8bb5-4b26-a2d4-8716e32a2349 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:01:00.087248 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-6415d92f-8bb5-4b26-a2d4-8716e32a2349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:01:00.181949 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 543/1104] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 14 03:01:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:01:00.338476 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097eae33-3877-4777-8464-5360ba5f0390 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:01:00.355340 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-097eae33-3877-4777-8464-5360ba5f0390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:01:00.422601 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:01:00.483097 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097eae33-3877-4777-8464-5360ba5f0390 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:01:00.594116 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772f6b60-610d-4b4d-85cf-bec33c36f998 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b07bf0faca47b28b50990d1720853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:01:00.594749 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772f6b60-610d-4b4d-85cf-bec33c36f998 service nova] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:01:00.595029 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772f6b60-610d-4b4d-85cf-bec33c36f998 service nova] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:01:00.595302 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772f6b60-610d-4b4d-85cf-bec33c36f998 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:01:00.596906 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-772f6b60-610d-4b4d-85cf-bec33c36f998 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:01:00.644662 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:01:00.685250 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772f6b60-610d-4b4d-85cf-bec33c36f998 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:01:00.686868 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772f6b60-610d-4b4d-85cf-bec33c36f998 service nova] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:01:00.705637 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 562/1105] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:01:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:01:01.806311 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d462017e-b0aa-4c0a-8503-5d0e1f3534e7 None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:01:01.806883 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d462017e-b0aa-4c0a-8503-5d0e1f3534e7 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:01:01.806883 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d462017e-b0aa-4c0a-8503-5d0e1f3534e7 None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:01:01.808223 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 544/1106] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 14 03:01:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 03:01:01.814377 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5723ae9-a65d-4808-80e8-93b6045849fc None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:01:01.814812 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5723ae9-a65d-4808-80e8-93b6045849fc None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:01:01.814944 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5723ae9-a65d-4808-80e8-93b6045849fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:01:01.890921 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-b5723ae9-a65d-4808-80e8-93b6045849fc None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:01:01.892112 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-b5723ae9-a65d-4808-80e8-93b6045849fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:01:01.985350 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 563/1107] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 14 03:01:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:01:02.134451 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d462017e-b0aa-4c0a-8503-5d0e1f3534e7 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:01:02.135955 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-d462017e-b0aa-4c0a-8503-5d0e1f3534e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:01:02.184612 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:01:02.279030 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d462017e-b0aa-4c0a-8503-5d0e1f3534e7 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:01:02.391561 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f8a4e6-eab8-4f96-8c20-a7454725d515 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b07bf0faca47b28b50990d1720853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:01:02.392166 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f8a4e6-eab8-4f96-8c20-a7454725d515 service nova] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:01:02.392447 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f8a4e6-eab8-4f96-8c20-a7454725d515 service nova] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:01:02.392718 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f8a4e6-eab8-4f96-8c20-a7454725d515 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:01:02.394344 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-71f8a4e6-eab8-4f96-8c20-a7454725d515 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:01:02.401720 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:01:02.483027 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f8a4e6-eab8-4f96-8c20-a7454725d515 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:01:02.484810 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f8a4e6-eab8-4f96-8c20-a7454725d515 service nova] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:01:02.502384 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 545/1108] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:01:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:01:03.579904 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1746ff90-e2a3-4b87-ade1-fe62c8ddcb26 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:01:03.580323 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1746ff90-e2a3-4b87-ade1-fe62c8ddcb26 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:01:03.580451 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1746ff90-e2a3-4b87-ade1-fe62c8ddcb26 None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:01:03.581826 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 564/1109] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 14 03:01:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 03:01:03.588188 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7cebab-6bae-4d1a-8270-3d9be56add84 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:01:03.588598 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7cebab-6bae-4d1a-8270-3d9be56add84 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:01:03.588761 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7cebab-6bae-4d1a-8270-3d9be56add84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:01:03.663513 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-4e7cebab-6bae-4d1a-8270-3d9be56add84 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:01:03.664708 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-4e7cebab-6bae-4d1a-8270-3d9be56add84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:01:03.730813 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:01:03.759266 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 546/1110] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 14 03:01:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:01:03.908305 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1746ff90-e2a3-4b87-ade1-fe62c8ddcb26 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:01:03.909839 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-1746ff90-e2a3-4b87-ade1-fe62c8ddcb26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:01:03.927462 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:01:04.052651 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1746ff90-e2a3-4b87-ade1-fe62c8ddcb26 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:01:04.137256 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:01:04.162443 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30d9ff4-ebe6-49ab-a07a-fc030fbe0c5d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b07bf0faca47b28b50990d1720853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:01:04.163642 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d9ff4-ebe6-49ab-a07a-fc030fbe0c5d service nova] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:01:04.163920 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d9ff4-ebe6-49ab-a07a-fc030fbe0c5d service nova] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:01:04.164189 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d9ff4-ebe6-49ab-a07a-fc030fbe0c5d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:01:04.165808 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-b30d9ff4-ebe6-49ab-a07a-fc030fbe0c5d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:01:04.252425 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30d9ff4-ebe6-49ab-a07a-fc030fbe0c5d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:01:04.254029 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30d9ff4-ebe6-49ab-a07a-fc030fbe0c5d service nova] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:01:04.271467 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 565/1111] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:01:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:01:05.367517 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19fc64c-0b38-4707-97ee-0b6293f1349a None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:01:05.368064 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19fc64c-0b38-4707-97ee-0b6293f1349a None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:01:05.368064 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19fc64c-0b38-4707-97ee-0b6293f1349a None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:01:05.369414 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 547/1112] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 14 03:01:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 03:01:05.375672 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd67aed8-18f5-4b6a-9708-0b468c44cb1b None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:01:05.378350 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd67aed8-18f5-4b6a-9708-0b468c44cb1b None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:01:05.378350 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd67aed8-18f5-4b6a-9708-0b468c44cb1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:01:05.450521 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-fd67aed8-18f5-4b6a-9708-0b468c44cb1b None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:01:05.451711 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-fd67aed8-18f5-4b6a-9708-0b468c44cb1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:01:05.493004 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:01:05.546731 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 566/1113] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 14 03:01:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:01:05.694274 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19fc64c-0b38-4707-97ee-0b6293f1349a None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:01:05.695733 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-b19fc64c-0b38-4707-97ee-0b6293f1349a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:01:05.838668 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19fc64c-0b38-4707-97ee-0b6293f1349a None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:01:05.893522 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:01:05.951721 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b71847-c8cf-40f7-9070-6170d9ce0584 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b07bf0faca47b28b50990d1720853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:01:05.952264 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b71847-c8cf-40f7-9070-6170d9ce0584 service nova] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:01:05.952547 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b71847-c8cf-40f7-9070-6170d9ce0584 service nova] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:01:05.952822 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b71847-c8cf-40f7-9070-6170d9ce0584 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:01:05.954457 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-d8b71847-c8cf-40f7-9070-6170d9ce0584 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:01:06.043854 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b71847-c8cf-40f7-9070-6170d9ce0584 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:01:06.045436 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b71847-c8cf-40f7-9070-6170d9ce0584 service nova] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:01:06.062935 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 548/1114] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:01:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:01:07.168063 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1c62b2-965d-40b6-800e-fa6fe4d1197c None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:01:07.168630 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1c62b2-965d-40b6-800e-fa6fe4d1197c None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:01:07.168630 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1c62b2-965d-40b6-800e-fa6fe4d1197c None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:01:07.170129 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 567/1115] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 14 03:01:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 03:01:07.176140 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a597328-0230-462a-af93-024692f1f21d None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:01:07.176608 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a597328-0230-462a-af93-024692f1f21d None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:01:07.176721 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a597328-0230-462a-af93-024692f1f21d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:01:07.250379 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-2a597328-0230-462a-af93-024692f1f21d None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:01:07.251586 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-2a597328-0230-462a-af93-024692f1f21d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:01:07.253563 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:01:07.347110 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 549/1116] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 14 03:01:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:01:07.495426 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1c62b2-965d-40b6-800e-fa6fe4d1197c None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:01:07.496945 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-5d1c62b2-965d-40b6-800e-fa6fe4d1197c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:01:07.639349 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1c62b2-965d-40b6-800e-fa6fe4d1197c None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:01:07.665602 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:01:07.748865 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7293d0f0-587c-4043-92ce-e0ab8b37bc49 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b07bf0faca47b28b50990d1720853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:01:07.749457 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7293d0f0-587c-4043-92ce-e0ab8b37bc49 service nova] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:01:07.749734 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7293d0f0-587c-4043-92ce-e0ab8b37bc49 service nova] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:01:07.750033 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7293d0f0-587c-4043-92ce-e0ab8b37bc49 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:01:07.751650 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-7293d0f0-587c-4043-92ce-e0ab8b37bc49 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:01:07.838486 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7293d0f0-587c-4043-92ce-e0ab8b37bc49 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:01:07.840064 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7293d0f0-587c-4043-92ce-e0ab8b37bc49 service nova] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:01:07.857872 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 568/1117] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:01:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:01:09.901974 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85861eb1-18f5-4807-9d3f-2c2c5f269cba None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:01:09.902476 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85861eb1-18f5-4807-9d3f-2c2c5f269cba None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:01:09.902558 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85861eb1-18f5-4807-9d3f-2c2c5f269cba None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:01:09.903849 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 550/1118] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 14 03:01:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 03:01:09.909299 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1aa03d-532b-44ea-bdf0-1c8951f7825b None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:01:09.909713 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1aa03d-532b-44ea-bdf0-1c8951f7825b None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:01:09.909875 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1aa03d-532b-44ea-bdf0-1c8951f7825b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:01:09.950445 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:01:09.976037 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-6c1aa03d-532b-44ea-bdf0-1c8951f7825b None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:01:09.977239 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-6c1aa03d-532b-44ea-bdf0-1c8951f7825b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:01:10.072090 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 569/1119] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 14 03:01:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:01:10.358378 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85861eb1-18f5-4807-9d3f-2c2c5f269cba None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:01:10.360222 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-85861eb1-18f5-4807-9d3f-2c2c5f269cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:01:10.461411 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85861eb1-18f5-4807-9d3f-2c2c5f269cba None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:01:10.493565 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:01:10.534871 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53e5927-955e-4b49-8fed-e176da8ec384 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:01:10.535489 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53e5927-955e-4b49-8fed-e176da8ec384 service neutron] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:01:10.535750 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53e5927-955e-4b49-8fed-e176da8ec384 service neutron] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:01:10.536020 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53e5927-955e-4b49-8fed-e176da8ec384 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:01:10.537634 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-d53e5927-955e-4b49-8fed-e176da8ec384 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:01:10.642772 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53e5927-955e-4b49-8fed-e176da8ec384 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:01:10.644095 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53e5927-955e-4b49-8fed-e176da8ec384 service neutron] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:01:10.662574 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 551/1120] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:01:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:01:12.938929 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd26b210-d383-4117-bed8-6cbfc81b67cd None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:01:12.939519 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd26b210-d383-4117-bed8-6cbfc81b67cd None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:01:12.939519 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd26b210-d383-4117-bed8-6cbfc81b67cd None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:01:12.940830 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 570/1121] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 14 03:01:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 03:01:12.946878 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c71f400-8cd0-4789-b83e-4dea581712b7 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:01:12.947271 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c71f400-8cd0-4789-b83e-4dea581712b7 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:01:12.947442 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c71f400-8cd0-4789-b83e-4dea581712b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:01:13.012263 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-7c71f400-8cd0-4789-b83e-4dea581712b7 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:01:13.013464 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-7c71f400-8cd0-4789-b83e-4dea581712b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:01:13.026013 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:01:13.108198 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 552/1122] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 14 03:01:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:01:13.392961 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd26b210-d383-4117-bed8-6cbfc81b67cd None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:01:13.394605 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-dd26b210-d383-4117-bed8-6cbfc81b67cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:01:13.497919 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd26b210-d383-4117-bed8-6cbfc81b67cd None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:01:13.510713 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:01:13.571430 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb60846b-e1d0-46b3-97e4-98d699da1a1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:01:13.572031 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb60846b-e1d0-46b3-97e4-98d699da1a1d service neutron] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:01:13.572310 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb60846b-e1d0-46b3-97e4-98d699da1a1d service neutron] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:01:13.572634 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb60846b-e1d0-46b3-97e4-98d699da1a1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:01:13.574258 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-cb60846b-e1d0-46b3-97e4-98d699da1a1d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:01:13.682298 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb60846b-e1d0-46b3-97e4-98d699da1a1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:01:13.683655 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb60846b-e1d0-46b3-97e4-98d699da1a1d service neutron] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:01:13.701514 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 571/1123] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:01:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:01:16.207209 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda63d1c-7445-46dc-8ac0-6a534ae48bb1 None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:01:16.207762 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda63d1c-7445-46dc-8ac0-6a534ae48bb1 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:01:16.207762 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda63d1c-7445-46dc-8ac0-6a534ae48bb1 None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:01:16.209615 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 553/1124] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 14 03:01:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 03:01:16.214994 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b09543-5abe-4c5b-bf9d-483837aaba09 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:01:16.215420 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b09543-5abe-4c5b-bf9d-483837aaba09 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:01:16.215525 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b09543-5abe-4c5b-bf9d-483837aaba09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:01:16.262324 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:01:16.280148 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-e4b09543-5abe-4c5b-bf9d-483837aaba09 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:01:16.283226 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-e4b09543-5abe-4c5b-bf9d-483837aaba09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:01:16.375273 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 572/1125] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 14 03:01:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:01:16.655263 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda63d1c-7445-46dc-8ac0-6a534ae48bb1 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:01:16.656790 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-fda63d1c-7445-46dc-8ac0-6a534ae48bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:01:16.757400 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda63d1c-7445-46dc-8ac0-6a534ae48bb1 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:01:16.795622 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:01:16.829695 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5098b50f-0bc3-4ef9-93f0-ee668899ce0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:01:16.830310 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5098b50f-0bc3-4ef9-93f0-ee668899ce0c service neutron] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:01:16.830587 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5098b50f-0bc3-4ef9-93f0-ee668899ce0c service neutron] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:01:16.830857 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5098b50f-0bc3-4ef9-93f0-ee668899ce0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:01:16.832424 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-5098b50f-0bc3-4ef9-93f0-ee668899ce0c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:01:16.937752 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5098b50f-0bc3-4ef9-93f0-ee668899ce0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:01:16.939162 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5098b50f-0bc3-4ef9-93f0-ee668899ce0c service neutron] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:01:16.957910 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 554/1126] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:01:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:01:19.208088 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a649ef0-1ca4-4c41-aba8-f79bc87de89b None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:01:19.208640 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a649ef0-1ca4-4c41-aba8-f79bc87de89b None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:01:19.208640 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a649ef0-1ca4-4c41-aba8-f79bc87de89b None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:01:19.210067 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 573/1127] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 14 03:01:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 03:01:19.215434 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb836be-4109-4d15-bdb4-ff59de16e16a None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:01:19.215874 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb836be-4109-4d15-bdb4-ff59de16e16a None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:01:19.215968 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb836be-4109-4d15-bdb4-ff59de16e16a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:01:19.280250 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-8eb836be-4109-4d15-bdb4-ff59de16e16a None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:01:19.281354 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-8eb836be-4109-4d15-bdb4-ff59de16e16a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:01:19.301553 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:01:19.376884 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 555/1128] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 14 03:01:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:01:19.665001 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a649ef0-1ca4-4c41-aba8-f79bc87de89b None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:01:19.666607 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-4a649ef0-1ca4-4c41-aba8-f79bc87de89b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:01:19.766849 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a649ef0-1ca4-4c41-aba8-f79bc87de89b None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:01:19.787719 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:01:19.839494 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f56a30-b37c-4f19-9d1d-052a317bc5ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:01:19.840003 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f56a30-b37c-4f19-9d1d-052a317bc5ac service neutron] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:01:19.840281 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f56a30-b37c-4f19-9d1d-052a317bc5ac service neutron] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:01:19.840551 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f56a30-b37c-4f19-9d1d-052a317bc5ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:01:19.842219 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-37f56a30-b37c-4f19-9d1d-052a317bc5ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:01:19.948786 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f56a30-b37c-4f19-9d1d-052a317bc5ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:01:19.950309 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f56a30-b37c-4f19-9d1d-052a317bc5ac service neutron] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:01:19.968463 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 574/1129] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:01:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:01:22.169353 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40401f4-7485-4bcd-89a2-708010c95ffb None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:01:22.169910 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40401f4-7485-4bcd-89a2-708010c95ffb None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:01:22.170005 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40401f4-7485-4bcd-89a2-708010c95ffb None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:01:22.171293 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 556/1130] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 14 03:01:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 03:01:22.176530 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a2e11e-d5f1-4f76-b478-751a8c8ecd75 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:01:22.176965 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a2e11e-d5f1-4f76-b478-751a8c8ecd75 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:01:22.177091 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a2e11e-d5f1-4f76-b478-751a8c8ecd75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:01:22.240429 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:01:22.243034 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-84a2e11e-d5f1-4f76-b478-751a8c8ecd75 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:01:22.244245 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-84a2e11e-d5f1-4f76-b478-751a8c8ecd75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:01:22.338915 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 575/1131] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 14 03:01:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:01:22.622245 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40401f4-7485-4bcd-89a2-708010c95ffb None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:01:22.623788 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-d40401f4-7485-4bcd-89a2-708010c95ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:01:22.725101 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40401f4-7485-4bcd-89a2-708010c95ffb None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:01:22.772930 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:01:22.797113 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b978a521-f380-4203-9984-197dc1988561 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:01:22.797711 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b978a521-f380-4203-9984-197dc1988561 service neutron] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:01:22.798023 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b978a521-f380-4203-9984-197dc1988561 service neutron] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:01:22.798297 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b978a521-f380-4203-9984-197dc1988561 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:01:22.799887 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-b978a521-f380-4203-9984-197dc1988561 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:01:22.907283 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b978a521-f380-4203-9984-197dc1988561 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:01:22.908636 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b978a521-f380-4203-9984-197dc1988561 service neutron] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:01:22.927342 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 557/1132] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:01:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:01:25.121574 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb418b-a8c8-4cae-b1a2-30a19d21eeac None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:01:25.122158 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb418b-a8c8-4cae-b1a2-30a19d21eeac None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:01:25.122158 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb418b-a8c8-4cae-b1a2-30a19d21eeac None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:01:25.123517 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 576/1133] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 14 03:01:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 03:01:25.128802 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a810504-984c-469e-9373-9a74bb562ddf None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:01:25.129200 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a810504-984c-469e-9373-9a74bb562ddf None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:01:25.129348 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a810504-984c-469e-9373-9a74bb562ddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:01:25.194162 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-7a810504-984c-469e-9373-9a74bb562ddf None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:01:25.195370 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-7a810504-984c-469e-9373-9a74bb562ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:01:25.223655 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:01:25.290065 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 558/1134] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 14 03:01:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:01:25.572280 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effb418b-a8c8-4cae-b1a2-30a19d21eeac None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:01:25.573848 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-effb418b-a8c8-4cae-b1a2-30a19d21eeac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:01:25.675813 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effb418b-a8c8-4cae-b1a2-30a19d21eeac None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:01:25.703660 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:01:25.749331 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe7d7f0-2125-4b8b-88db-fd82832ff787 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:01:25.749965 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe7d7f0-2125-4b8b-88db-fd82832ff787 service neutron] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:01:25.750248 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe7d7f0-2125-4b8b-88db-fd82832ff787 service neutron] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:01:25.750704 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe7d7f0-2125-4b8b-88db-fd82832ff787 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:01:25.752321 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-afe7d7f0-2125-4b8b-88db-fd82832ff787 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:01:25.859450 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe7d7f0-2125-4b8b-88db-fd82832ff787 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:01:25.860885 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe7d7f0-2125-4b8b-88db-fd82832ff787 service neutron] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:01:25.878469 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 577/1135] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:01:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:01:28.060082 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6d4803-dd89-439e-8a64-967141882469 None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:01:28.060572 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6d4803-dd89-439e-8a64-967141882469 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:01:28.060682 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6d4803-dd89-439e-8a64-967141882469 None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:01:28.062014 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 559/1136] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 14 03:01:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 03:01:28.067339 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a354036-04fb-4d1d-b92e-f456a273c51f None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:01:28.067746 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a354036-04fb-4d1d-b92e-f456a273c51f None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:01:28.068500 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a354036-04fb-4d1d-b92e-f456a273c51f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:01:28.134538 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-8a354036-04fb-4d1d-b92e-f456a273c51f None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:01:28.135727 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-8a354036-04fb-4d1d-b92e-f456a273c51f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:01:28.143890 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:01:28.233233 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 578/1137] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 14 03:01:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 03:01:28.514846 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6d4803-dd89-439e-8a64-967141882469 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:01:28.516426 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-3e6d4803-dd89-439e-8a64-967141882469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:01:28.618903 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6d4803-dd89-439e-8a64-967141882469 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:01:28.672630 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:01:28.690266 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c042fe-f807-4d97-9e43-152ea5368e04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:01:28.690864 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c042fe-f807-4d97-9e43-152ea5368e04 service neutron] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:01:28.691140 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c042fe-f807-4d97-9e43-152ea5368e04 service neutron] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:01:28.691410 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c042fe-f807-4d97-9e43-152ea5368e04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:01:28.693073 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-d6c042fe-f807-4d97-9e43-152ea5368e04 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:01:28.802003 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c042fe-f807-4d97-9e43-152ea5368e04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:01:28.804302 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c042fe-f807-4d97-9e43-152ea5368e04 service neutron] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:01:28.823772 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 560/1138] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:01:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:01:31.004922 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d57c0e5-44f1-4b7b-93eb-834133d59095 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:01:31.005453 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d57c0e5-44f1-4b7b-93eb-834133d59095 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:01:31.005572 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d57c0e5-44f1-4b7b-93eb-834133d59095 None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:01:31.007408 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 579/1139] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 14 03:01:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 03:01:31.013608 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dc2989-5372-46a4-ac8e-80eeb5ecfc4c None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:01:31.014187 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dc2989-5372-46a4-ac8e-80eeb5ecfc4c None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:01:31.014294 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dc2989-5372-46a4-ac8e-80eeb5ecfc4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:01:31.086106 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-57dc2989-5372-46a4-ac8e-80eeb5ecfc4c None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:01:31.087313 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-57dc2989-5372-46a4-ac8e-80eeb5ecfc4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:01:31.126011 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:01:31.183057 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 561/1140] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 14 03:01:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:01:31.466068 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d57c0e5-44f1-4b7b-93eb-834133d59095 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:01:31.467599 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-2d57c0e5-44f1-4b7b-93eb-834133d59095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:01:31.568345 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d57c0e5-44f1-4b7b-93eb-834133d59095 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:01:31.605370 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:01:31.639948 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219b9355-b377-4559-9cbe-c72b8989f269 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:01:31.640556 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219b9355-b377-4559-9cbe-c72b8989f269 service neutron] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:01:31.640837 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219b9355-b377-4559-9cbe-c72b8989f269 service neutron] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:01:31.641107 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219b9355-b377-4559-9cbe-c72b8989f269 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:01:31.642716 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-219b9355-b377-4559-9cbe-c72b8989f269 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:01:31.748107 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219b9355-b377-4559-9cbe-c72b8989f269 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:01:31.749473 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219b9355-b377-4559-9cbe-c72b8989f269 service neutron] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:01:31.767569 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 580/1141] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:01:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:01:33.951885 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35883e51-a456-445a-b630-5b4ecab3a2a9 None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:01:33.952403 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35883e51-a456-445a-b630-5b4ecab3a2a9 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:01:33.952403 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35883e51-a456-445a-b630-5b4ecab3a2a9 None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:01:33.953711 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 562/1142] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 14 03:01:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 03:01:33.959540 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75516817-7821-4e84-bde9-daeb7dfa89aa None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:01:33.959953 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75516817-7821-4e84-bde9-daeb7dfa89aa None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:01:33.960102 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75516817-7821-4e84-bde9-daeb7dfa89aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:01:34.026800 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-75516817-7821-4e84-bde9-daeb7dfa89aa None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:01:34.028009 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-75516817-7821-4e84-bde9-daeb7dfa89aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:01:34.046215 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:01:34.123160 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 581/1143] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 14 03:01:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:01:34.407804 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35883e51-a456-445a-b630-5b4ecab3a2a9 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:01:34.409389 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-35883e51-a456-445a-b630-5b4ecab3a2a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:01:34.511412 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35883e51-a456-445a-b630-5b4ecab3a2a9 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:01:34.576336 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:01:34.583793 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147dff7a-6466-428f-954b-3fbe84cb4630 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:01:34.585473 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147dff7a-6466-428f-954b-3fbe84cb4630 service neutron] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:01:34.585473 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147dff7a-6466-428f-954b-3fbe84cb4630 service neutron] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:01:34.585473 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147dff7a-6466-428f-954b-3fbe84cb4630 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:01:34.586517 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-147dff7a-6466-428f-954b-3fbe84cb4630 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:01:34.695872 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147dff7a-6466-428f-954b-3fbe84cb4630 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:01:34.697218 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147dff7a-6466-428f-954b-3fbe84cb4630 service neutron] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:01:34.714594 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 563/1144] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:01:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:01:36.860040 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d925f9d-6db9-4552-b6ce-9674b9fdbe45 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:01:36.860656 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d925f9d-6db9-4552-b6ce-9674b9fdbe45 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:01:36.860766 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d925f9d-6db9-4552-b6ce-9674b9fdbe45 None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:01:36.862151 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 582/1145] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 14 03:01:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 03:01:36.868177 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c04675-4197-4d4a-a1dd-0bb7964e9321 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:01:36.868589 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c04675-4197-4d4a-a1dd-0bb7964e9321 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:01:36.868731 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c04675-4197-4d4a-a1dd-0bb7964e9321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:01:36.933865 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-53c04675-4197-4d4a-a1dd-0bb7964e9321 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:01:36.935105 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-53c04675-4197-4d4a-a1dd-0bb7964e9321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:01:36.982991 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:01:37.030867 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 564/1146] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 14 03:01:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:01:37.312278 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d925f9d-6db9-4552-b6ce-9674b9fdbe45 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:01:37.313835 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-0d925f9d-6db9-4552-b6ce-9674b9fdbe45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:01:37.415504 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d925f9d-6db9-4552-b6ce-9674b9fdbe45 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:01:37.462895 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:01:37.487310 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581eb643-9a8f-4280-86c3-4704aa4737f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:01:37.487941 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581eb643-9a8f-4280-86c3-4704aa4737f3 service neutron] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:01:37.488402 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581eb643-9a8f-4280-86c3-4704aa4737f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:01:37.488839 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581eb643-9a8f-4280-86c3-4704aa4737f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:01:37.490738 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-581eb643-9a8f-4280-86c3-4704aa4737f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:01:37.596083 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581eb643-9a8f-4280-86c3-4704aa4737f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:01:37.597519 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581eb643-9a8f-4280-86c3-4704aa4737f3 service neutron] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:01:37.615811 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 583/1147] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:01:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:01:47.029254 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=22 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:01:47.613570 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=66 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:01:57.393742 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c651e8d-5132-4518-9132-9d2d6ce813df None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:01:57.394298 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c651e8d-5132-4518-9132-9d2d6ce813df None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:01:57.394298 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c651e8d-5132-4518-9132-9d2d6ce813df None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:01:57.396179 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 565/1148] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 03:01:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 03:01:57.401360 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bc0fd8-53d1-4dd7-bf1b-13f01d64233f None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:01:57.401819 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bc0fd8-53d1-4dd7-bf1b-13f01d64233f None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:01:57.401951 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bc0fd8-53d1-4dd7-bf1b-13f01d64233f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:01:57.466538 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-f9bc0fd8-53d1-4dd7-bf1b-13f01d64233f None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:01:57.467739 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-f9bc0fd8-53d1-4dd7-bf1b-13f01d64233f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:01:57.562295 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 584/1149] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 03:01:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:01:57.854071 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477e20a9-707b-4b57-8ebb-4bb1eaba4a95 None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:01:57.854357 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477e20a9-707b-4b57-8ebb-4bb1eaba4a95 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:01:57.854633 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477e20a9-707b-4b57-8ebb-4bb1eaba4a95 None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:01:57.855977 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 566/1150] 10.222.0.39 () {54 vars in 962 bytes} [Fri Oct 14 03:01:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 03:01:57.861204 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a026a839-941c-430a-ab06-db345c0c262c None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:01:57.861520 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a026a839-941c-430a-ab06-db345c0c262c None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:01:57.861829 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a026a839-941c-430a-ab06-db345c0c262c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:01:57.936325 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-a026a839-941c-430a-ab06-db345c0c262c None None] MFA Rules not processed for user `6a3cd1fbe6df48f8aa90d4ea69905e9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:01:57.937526 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-a026a839-941c-430a-ab06-db345c0c262c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:01:57.958007 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:01:58.012858 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 585/1151] 10.222.0.39 () {58 vars in 1059 bytes} [Fri Oct 14 03:01:57 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:01:58.020593 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477e20a9-707b-4b57-8ebb-4bb1eaba4a95 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:01:58.022142 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-477e20a9-707b-4b57-8ebb-4bb1eaba4a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:01:58.123285 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477e20a9-707b-4b57-8ebb-4bb1eaba4a95 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:01:58.197836 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927ad8b2-1e37-4ce9-b18c-df5f90ead285 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a3cd1fbe6df48f8aa90d4ea69905e9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:01:58.198439 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927ad8b2-1e37-4ce9-b18c-df5f90ead285 service glance] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:01:58.198718 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927ad8b2-1e37-4ce9-b18c-df5f90ead285 service glance] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:01:58.198986 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927ad8b2-1e37-4ce9-b18c-df5f90ead285 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:01:58.200563 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-927ad8b2-1e37-4ce9-b18c-df5f90ead285 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:01:58.255403 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:01:58.311313 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927ad8b2-1e37-4ce9-b18c-df5f90ead285 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:01:58.312739 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927ad8b2-1e37-4ce9-b18c-df5f90ead285 service glance] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:01:58.330736 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 567/1152] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:01:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:02:00.557397 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c45d78-4af7-4e50-820f-9991d7a4820a None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:02:00.557973 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c45d78-4af7-4e50-820f-9991d7a4820a None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:02:00.557973 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c45d78-4af7-4e50-820f-9991d7a4820a None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:02:00.559306 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 586/1153] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 03:02:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 03:02:00.564642 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e2199a-5ce8-4f64-a562-3974af937aec None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:02:00.565054 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e2199a-5ce8-4f64-a562-3974af937aec None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:02:00.565176 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e2199a-5ce8-4f64-a562-3974af937aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:02:00.629653 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-e5e2199a-5ce8-4f64-a562-3974af937aec None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:02:00.630873 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-e5e2199a-5ce8-4f64-a562-3974af937aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:02:00.724142 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 568/1154] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 03:02:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:02:01.000498 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c45d78-4af7-4e50-820f-9991d7a4820a None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:02:01.002076 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-21c45d78-4af7-4e50-820f-9991d7a4820a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:02:01.125229 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c45d78-4af7-4e50-820f-9991d7a4820a None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:02:01.173515 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:02:01.217055 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e258597-4e0f-415b-b0f5-de5a5c8388e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:02:01.217653 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e258597-4e0f-415b-b0f5-de5a5c8388e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:02:01.217975 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e258597-4e0f-415b-b0f5-de5a5c8388e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:02:01.218246 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e258597-4e0f-415b-b0f5-de5a5c8388e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:02:01.218879 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e258597-4e0f-415b-b0f5-de5a5c8388e3 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:02:01.219983 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: WARNING py.warnings [None req-0e258597-4e0f-415b-b0f5-de5a5c8388e3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 14 03:02:01.219983 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: warnings.warn(msg) Oct 14 03:02:01.219983 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: Oct 14 03:02:01.221122 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e258597-4e0f-415b-b0f5-de5a5c8388e3 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:02:01.240310 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 587/1155] 10.222.0.39 () {56 vars in 1108 bytes} [Fri Oct 14 03:02:00 2022] GET /identity/v3/projects => generated 2553 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:02:03.428618 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9e8c8b-5ef7-480d-9ad2-d05574d87b5f None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:02:03.429220 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9e8c8b-5ef7-480d-9ad2-d05574d87b5f None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:02:03.429220 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9e8c8b-5ef7-480d-9ad2-d05574d87b5f None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:02:03.430612 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 569/1156] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 03:02:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 03:02:03.435922 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bd9ca1-91cf-41ca-b0ee-2d811ef627bc None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:02:03.436357 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bd9ca1-91cf-41ca-b0ee-2d811ef627bc None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:02:03.436484 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bd9ca1-91cf-41ca-b0ee-2d811ef627bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:02:03.502692 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-48bd9ca1-91cf-41ca-b0ee-2d811ef627bc None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:02:03.503903 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-48bd9ca1-91cf-41ca-b0ee-2d811ef627bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:02:03.599650 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 588/1157] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 03:02:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:02:03.892994 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9e8c8b-5ef7-480d-9ad2-d05574d87b5f None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:02:03.895182 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-7f9e8c8b-5ef7-480d-9ad2-d05574d87b5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:02:03.904902 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:02:04.039985 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9e8c8b-5ef7-480d-9ad2-d05574d87b5f None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:02:04.119012 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:02:04.151349 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81b1597-606a-4b86-a5c7-d9d3f1d34cfc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b07bf0faca47b28b50990d1720853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:02:04.151893 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81b1597-606a-4b86-a5c7-d9d3f1d34cfc service nova] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:02:04.152179 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81b1597-606a-4b86-a5c7-d9d3f1d34cfc service nova] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:02:04.152459 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81b1597-606a-4b86-a5c7-d9d3f1d34cfc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:02:04.154102 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-f81b1597-606a-4b86-a5c7-d9d3f1d34cfc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:02:04.243432 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81b1597-606a-4b86-a5c7-d9d3f1d34cfc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:02:04.245064 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81b1597-606a-4b86-a5c7-d9d3f1d34cfc service nova] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:02:04.263166 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 570/1158] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:02:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:02:06.558285 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53622c81-31a7-4726-9495-c8c328362ebc None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:02:06.558853 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53622c81-31a7-4726-9495-c8c328362ebc None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:02:06.558853 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53622c81-31a7-4726-9495-c8c328362ebc None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:02:06.560358 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 589/1159] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 03:02:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 03:02:06.565500 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c264500-63dd-4601-a10b-faedce55f6dc None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:02:06.565947 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c264500-63dd-4601-a10b-faedce55f6dc None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:02:06.566603 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c264500-63dd-4601-a10b-faedce55f6dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:02:06.631215 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-1c264500-63dd-4601-a10b-faedce55f6dc None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:02:06.632418 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-1c264500-63dd-4601-a10b-faedce55f6dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:02:06.677407 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:02:06.727094 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 571/1160] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 03:02:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:02:07.011131 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53622c81-31a7-4726-9495-c8c328362ebc None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:02:07.012663 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-53622c81-31a7-4726-9495-c8c328362ebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:02:07.035809 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:02:07.113852 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53622c81-31a7-4726-9495-c8c328362ebc None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:02:07.186038 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a68a8a4-c8b3-451e-bf59-f66b97ee4b63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:02:07.186641 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a68a8a4-c8b3-451e-bf59-f66b97ee4b63 service neutron] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:02:07.186920 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a68a8a4-c8b3-451e-bf59-f66b97ee4b63 service neutron] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:02:07.187190 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a68a8a4-c8b3-451e-bf59-f66b97ee4b63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:02:07.188781 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-2a68a8a4-c8b3-451e-bf59-f66b97ee4b63 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:02:07.261469 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:02:07.295959 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a68a8a4-c8b3-451e-bf59-f66b97ee4b63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:02:07.297391 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a68a8a4-c8b3-451e-bf59-f66b97ee4b63 service neutron] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:02:07.315395 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 590/1161] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:02:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:02:09.469302 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c37a2f3-f6fc-4334-af37-e0d1045658a5 None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:02:09.469876 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c37a2f3-f6fc-4334-af37-e0d1045658a5 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:02:09.469876 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c37a2f3-f6fc-4334-af37-e0d1045658a5 None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:02:09.471245 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 572/1162] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 03:02:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 03:02:09.476948 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e863e9d2-006b-430b-ba73-e2f2d5cd1bad None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:02:09.477350 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e863e9d2-006b-430b-ba73-e2f2d5cd1bad None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:02:09.477488 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e863e9d2-006b-430b-ba73-e2f2d5cd1bad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:02:09.542294 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-e863e9d2-006b-430b-ba73-e2f2d5cd1bad None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:02:09.543487 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-e863e9d2-006b-430b-ba73-e2f2d5cd1bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:02:09.636627 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 591/1163] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 03:02:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:02:09.920852 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c37a2f3-f6fc-4334-af37-e0d1045658a5 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:02:09.923034 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-5c37a2f3-f6fc-4334-af37-e0d1045658a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:02:10.027722 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c37a2f3-f6fc-4334-af37-e0d1045658a5 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:02:10.097768 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:02:10.099201 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841da26c-0b14-4e02-b153-5c7ce5b0653b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:02:10.099792 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841da26c-0b14-4e02-b153-5c7ce5b0653b service neutron] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:02:10.100070 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841da26c-0b14-4e02-b153-5c7ce5b0653b service neutron] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:02:10.100341 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841da26c-0b14-4e02-b153-5c7ce5b0653b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:02:10.101949 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-841da26c-0b14-4e02-b153-5c7ce5b0653b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:02:10.209441 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841da26c-0b14-4e02-b153-5c7ce5b0653b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:02:10.210830 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841da26c-0b14-4e02-b153-5c7ce5b0653b service neutron] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:02:10.228559 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 573/1164] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:02:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:02:12.538047 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b4fc00-903e-496a-b05c-5fbff488269d None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:02:12.538621 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b4fc00-903e-496a-b05c-5fbff488269d None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:02:12.538621 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b4fc00-903e-496a-b05c-5fbff488269d None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:02:12.539941 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 592/1165] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 03:02:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 03:02:12.545406 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da7211f-4e48-4303-9e18-6b77a059c06b None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:02:12.545851 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da7211f-4e48-4303-9e18-6b77a059c06b None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:02:12.545990 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da7211f-4e48-4303-9e18-6b77a059c06b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:02:12.610494 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-1da7211f-4e48-4303-9e18-6b77a059c06b None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:02:12.611706 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-1da7211f-4e48-4303-9e18-6b77a059c06b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:02:12.663839 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:02:12.707084 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 574/1166] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 03:02:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:02:12.995908 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b4fc00-903e-496a-b05c-5fbff488269d None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:02:12.997433 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-68b4fc00-903e-496a-b05c-5fbff488269d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:02:13.033752 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:02:13.101733 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b4fc00-903e-496a-b05c-5fbff488269d None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:02:13.174438 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f92aeea-dca7-4728-869d-a2cc459bf818 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:02:13.175044 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f92aeea-dca7-4728-869d-a2cc459bf818 service neutron] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:02:13.175327 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f92aeea-dca7-4728-869d-a2cc459bf818 service neutron] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:02:13.175600 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f92aeea-dca7-4728-869d-a2cc459bf818 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:02:13.177731 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-0f92aeea-dca7-4728-869d-a2cc459bf818 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:02:13.256124 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:02:13.284282 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f92aeea-dca7-4728-869d-a2cc459bf818 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:02:13.285707 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f92aeea-dca7-4728-869d-a2cc459bf818 service neutron] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:02:13.304233 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 593/1167] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:02:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:02:15.843138 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318dad05-b941-4d68-979d-624d3299cce7 None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:02:15.843668 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318dad05-b941-4d68-979d-624d3299cce7 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:02:15.843668 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318dad05-b941-4d68-979d-624d3299cce7 None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:02:15.845014 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 575/1168] 10.222.0.39 () {54 vars in 927 bytes} [Fri Oct 14 03:02:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 03:02:15.850375 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bcaaf4-bad6-4fc2-a117-d2527e489fba None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:02:15.850797 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bcaaf4-bad6-4fc2-a117-d2527e489fba None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:02:15.850996 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bcaaf4-bad6-4fc2-a117-d2527e489fba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:02:15.917390 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-f3bcaaf4-bad6-4fc2-a117-d2527e489fba None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:02:15.918559 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-f3bcaaf4-bad6-4fc2-a117-d2527e489fba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:02:16.015504 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 594/1169] 10.222.0.39 () {58 vars in 1024 bytes} [Fri Oct 14 03:02:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:02:16.306126 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318dad05-b941-4d68-979d-624d3299cce7 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:02:16.307657 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-318dad05-b941-4d68-979d-624d3299cce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:02:16.409928 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318dad05-b941-4d68-979d-624d3299cce7 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:02:16.482420 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a2393f-2360-4216-b0ec-44b06f67a6f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:02:16.483011 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a2393f-2360-4216-b0ec-44b06f67a6f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:02:16.483289 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a2393f-2360-4216-b0ec-44b06f67a6f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:02:16.483558 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a2393f-2360-4216-b0ec-44b06f67a6f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:02:16.485381 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-91a2393f-2360-4216-b0ec-44b06f67a6f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:02:16.494225 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:02:16.592738 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a2393f-2360-4216-b0ec-44b06f67a6f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:02:16.594204 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a2393f-2360-4216-b0ec-44b06f67a6f0 service neutron] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:02:16.612173 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 576/1170] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:02:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:02:26.012299 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=82 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:02:26.609977 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=50 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:03:59.395360 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f7ded1-2f87-4ac6-b354-5a9f5a3384e6 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:03:59.395955 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f7ded1-2f87-4ac6-b354-5a9f5a3384e6 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:03:59.395955 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f7ded1-2f87-4ac6-b354-5a9f5a3384e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:03:59.476036 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-97f7ded1-2f87-4ac6-b354-5a9f5a3384e6 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:03:59.477243 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-97f7ded1-2f87-4ac6-b354-5a9f5a3384e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:03:59.580043 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 595/1171] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:03:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:03:59.587885 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3801d2f-0e4e-4a18-b298-d878e24cac5c None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:03:59.588443 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3801d2f-0e4e-4a18-b298-d878e24cac5c None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:03:59.588443 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3801d2f-0e4e-4a18-b298-d878e24cac5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:03:59.607707 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-f3801d2f-0e4e-4a18-b298-d878e24cac5c None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:03:59.608891 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-f3801d2f-0e4e-4a18-b298-d878e24cac5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:03:59.617454 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 577/1172] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:03:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:03:59.621820 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f7ded1-2f87-4ac6-b354-5a9f5a3384e6 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:03:59.631830 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f7ded1-2f87-4ac6-b354-5a9f5a3384e6 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:03:59.636890 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884859e6-498d-4f0f-9e26-7eb5a528feeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:03:59.637556 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884859e6-498d-4f0f-9e26-7eb5a528feeb admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:03:59.637902 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884859e6-498d-4f0f-9e26-7eb5a528feeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:03:59.638190 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884859e6-498d-4f0f-9e26-7eb5a528feeb admin admin] PATH_INFO: `/v3/domains` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:03:59.638847 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884859e6-498d-4f0f-9e26-7eb5a528feeb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 03:03:59.639161 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884859e6-498d-4f0f-9e26-7eb5a528feeb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:03:59.640190 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: WARNING py.warnings [None req-884859e6-498d-4f0f-9e26-7eb5a528feeb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 14 03:03:59.640190 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: warnings.warn(msg) Oct 14 03:03:59.640190 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: Oct 14 03:03:59.641109 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884859e6-498d-4f0f-9e26-7eb5a528feeb admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:03:59.653468 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 596/1173] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Oct 14 03:03:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 03:03:59.662813 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3801d2f-0e4e-4a18-b298-d878e24cac5c None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:03:59.672100 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3801d2f-0e4e-4a18-b298-d878e24cac5c None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:03:59.676742 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af91f1fa-2983-4b8f-9a6f-67d0c5da43c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:03:59.677338 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af91f1fa-2983-4b8f-9a6f-67d0c5da43c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:03:59.677620 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af91f1fa-2983-4b8f-9a6f-67d0c5da43c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:03:59.677938 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af91f1fa-2983-4b8f-9a6f-67d0c5da43c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:03:59.678571 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af91f1fa-2983-4b8f-9a6f-67d0c5da43c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:03:59.679665 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: WARNING py.warnings [None req-af91f1fa-2983-4b8f-9a6f-67d0c5da43c2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 14 03:03:59.679665 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: warnings.warn(msg) Oct 14 03:03:59.679665 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: Oct 14 03:03:59.680846 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af91f1fa-2983-4b8f-9a6f-67d0c5da43c2 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:03:59.760401 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 578/1174] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:03:59 2022] POST /identity/v3/projects => generated 377 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:03:59.764453 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884859e6-498d-4f0f-9e26-7eb5a528feeb admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:03:59.775108 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884859e6-498d-4f0f-9e26-7eb5a528feeb admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:03:59.824331 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:03:59.831212 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af68b225-300a-435d-b25c-f5b841e631d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:03:59.831817 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af68b225-300a-435d-b25c-f5b841e631d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:03:59.832105 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af68b225-300a-435d-b25c-f5b841e631d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:03:59.832383 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af68b225-300a-435d-b25c-f5b841e631d1 admin admin] PATH_INFO: `/v3/users` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:03:59.833030 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af68b225-300a-435d-b25c-f5b841e631d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:03:59.834199 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: WARNING py.warnings [None req-af68b225-300a-435d-b25c-f5b841e631d1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 14 03:03:59.834199 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: warnings.warn(msg) Oct 14 03:03:59.834199 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: Oct 14 03:03:59.835421 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af68b225-300a-435d-b25c-f5b841e631d1 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:03:59.902925 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 597/1175] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:03:59 2022] POST /identity/v3/users => generated 289 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:03:59.906901 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af91f1fa-2983-4b8f-9a6f-67d0c5da43c2 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:03:59.917236 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af91f1fa-2983-4b8f-9a6f-67d0c5da43c2 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:03:59.921949 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d8228e-1798-4197-a22f-436732bb9154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:03:59.932257 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d8228e-1798-4197-a22f-436732bb9154 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:03:59.932257 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d8228e-1798-4197-a22f-436732bb9154 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:03:59.932257 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d8228e-1798-4197-a22f-436732bb9154 admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:03:59.932257 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d8228e-1798-4197-a22f-436732bb9154 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:03:59.932257 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d8228e-1798-4197-a22f-436732bb9154 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:03:59.938413 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 579/1176] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:03:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:03:59.942820 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af68b225-300a-435d-b25c-f5b841e631d1 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:03:59.952955 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af68b225-300a-435d-b25c-f5b841e631d1 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:03:59.957581 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53b3c8e-9e31-493b-b4b8-0e9012d7e631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:03:59.958212 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53b3c8e-9e31-493b-b4b8-0e9012d7e631 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:03:59.958494 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53b3c8e-9e31-493b-b4b8-0e9012d7e631 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:03:59.958770 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53b3c8e-9e31-493b-b4b8-0e9012d7e631 admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:03:59.959412 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53b3c8e-9e31-493b-b4b8-0e9012d7e631 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:03:59.961021 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53b3c8e-9e31-493b-b4b8-0e9012d7e631 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:03:59.974299 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 598/1177] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:03:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:03:59.978430 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d8228e-1798-4197-a22f-436732bb9154 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:03:59.988890 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d8228e-1798-4197-a22f-436732bb9154 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:03:59.993525 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10940a6a-7f00-4483-8ec7-0b435d2bf3d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:03:59.994205 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10940a6a-7f00-4483-8ec7-0b435d2bf3d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:03:59.994484 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10940a6a-7f00-4483-8ec7-0b435d2bf3d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:03:59.994755 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10940a6a-7f00-4483-8ec7-0b435d2bf3d5 admin admin] PATH_INFO: `/v3/projects/dfdfef40759a48ba80b596f39ced5b47/users/ba0f8e00a48a4e99800737623db5de5f/roles/9bd0470b544240209a420f640c79c06c` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:04:00.011134 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10940a6a-7f00-4483-8ec7-0b435d2bf3d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfdfef40759a48ba80b596f39ced5b47, user_id=ba0f8e00a48a4e99800737623db5de5f, role_id=9bd0470b544240209a420f640c79c06c)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:04:00.013788 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10940a6a-7f00-4483-8ec7-0b435d2bf3d5 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:04:00.056450 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 580/1178] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:03:59 2022] PUT /identity/v3/projects/dfdfef40759a48ba80b596f39ced5b47/users/ba0f8e00a48a4e99800737623db5de5f/roles/9bd0470b544240209a420f640c79c06c => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:04:00.068273 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ed7c00-6de1-4411-97dd-53fc30c62e39 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:04:00.068599 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ed7c00-6de1-4411-97dd-53fc30c62e39 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:04:00.068882 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ed7c00-6de1-4411-97dd-53fc30c62e39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:04:00.095760 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-a0ed7c00-6de1-4411-97dd-53fc30c62e39 None None] MFA Rules not processed for user `ba0f8e00a48a4e99800737623db5de5f`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:04:00.096958 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-a0ed7c00-6de1-4411-97dd-53fc30c62e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:04:00.160192 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 599/1179] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:04:00 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:04:00.173714 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80648037-8197-4cdb-b2a4-97853d20e4aa None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:04:00.174030 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80648037-8197-4cdb-b2a4-97853d20e4aa None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:04:00.174307 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80648037-8197-4cdb-b2a4-97853d20e4aa None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:04:00.175624 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 581/1180] 10.222.0.39 () {54 vars in 963 bytes} [Fri Oct 14 03:04:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 03:04:00.180397 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db4cf71-ea07-4ba2-a529-002635a6a0f3 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:04:00.180688 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db4cf71-ea07-4ba2-a529-002635a6a0f3 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:04:00.180995 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db4cf71-ea07-4ba2-a529-002635a6a0f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:04:00.239958 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-5db4cf71-ea07-4ba2-a529-002635a6a0f3 None None] MFA Rules not processed for user `ab599d98b0624a2e89090312bcab713f`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:04:00.241146 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-5db4cf71-ea07-4ba2-a529-002635a6a0f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:04:00.315528 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 600/1181] 10.222.0.39 () {58 vars in 1060 bytes} [Fri Oct 14 03:04:00 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:04:00.323228 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80648037-8197-4cdb-b2a4-97853d20e4aa None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:04:00.333359 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80648037-8197-4cdb-b2a4-97853d20e4aa None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:04:00.336797 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9d96e0-314e-4814-b36f-9278dca98997 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab599d98b0624a2e89090312bcab713f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:04:00.337387 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9d96e0-314e-4814-b36f-9278dca98997 service cinder] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:04:00.337664 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9d96e0-314e-4814-b36f-9278dca98997 service cinder] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:04:00.337965 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9d96e0-314e-4814-b36f-9278dca98997 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:04:00.346618 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9d96e0-314e-4814-b36f-9278dca98997 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:04:00.348061 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9d96e0-314e-4814-b36f-9278dca98997 service cinder] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:04:00.356584 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 582/1182] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:04:00 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:04:00.409442 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc755f90-d824-4481-b3c3-62870f946c24 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:04:00.409881 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc755f90-d824-4481-b3c3-62870f946c24 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:04:00.410030 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc755f90-d824-4481-b3c3-62870f946c24 None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:04:00.411359 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 601/1183] 10.222.0.39 () {52 vars in 830 bytes} [Fri Oct 14 03:04:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 03:04:00.414940 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9d96e0-314e-4814-b36f-9278dca98997 service cinder] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:04:00.475374 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9d96e0-314e-4814-b36f-9278dca98997 service cinder] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:04:00.480008 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd76ebb-66f1-4cb9-806e-14e1a0823f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:04:00.480622 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd76ebb-66f1-4cb9-806e-14e1a0823f15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:04:00.480902 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd76ebb-66f1-4cb9-806e-14e1a0823f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:04:00.481173 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd76ebb-66f1-4cb9-806e-14e1a0823f15 admin admin] PATH_INFO: `/v3/users/ba0f8e00a48a4e99800737623db5de5f` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:04:00.482312 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd76ebb-66f1-4cb9-806e-14e1a0823f15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba0f8e00a48a4e99800737623db5de5f)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:04:00.483398 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: WARNING py.warnings [None req-dbd76ebb-66f1-4cb9-806e-14e1a0823f15 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 14 03:04:00.483398 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: warnings.warn(msg) Oct 14 03:04:00.483398 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: Oct 14 03:04:00.484852 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd76ebb-66f1-4cb9-806e-14e1a0823f15 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:04:00.694128 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-dbd76ebb-66f1-4cb9-806e-14e1a0823f15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba0f8e00a48a4e99800737623db5de5f'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:04:00.708810 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-dbd76ebb-66f1-4cb9-806e-14e1a0823f15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba0f8e00a48a4e99800737623db5de5f'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:04:00.732197 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-dbd76ebb-66f1-4cb9-806e-14e1a0823f15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba0f8e00a48a4e99800737623db5de5f'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:04:00.732886 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-dbd76ebb-66f1-4cb9-806e-14e1a0823f15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba0f8e00a48a4e99800737623db5de5f'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:04:00.737103 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:04:00.828105 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-dbd76ebb-66f1-4cb9-806e-14e1a0823f15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba0f8e00a48a4e99800737623db5de5f'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:04:00.834724 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 583/1184] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:04:00 2022] DELETE /identity/v3/users/ba0f8e00a48a4e99800737623db5de5f => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:04:00.839070 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47de26d8-bdbf-40a4-850d-4322ba2ac322 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:04:00.839365 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47de26d8-bdbf-40a4-850d-4322ba2ac322 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:04:00.839692 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47de26d8-bdbf-40a4-850d-4322ba2ac322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:04:00.859022 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-47de26d8-bdbf-40a4-850d-4322ba2ac322 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:04:00.860486 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-47de26d8-bdbf-40a4-850d-4322ba2ac322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:04:00.869626 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:04:00.918579 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 602/1185] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:04:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:04:00.928292 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd76ebb-66f1-4cb9-806e-14e1a0823f15 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:04:00.929693 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-dbd76ebb-66f1-4cb9-806e-14e1a0823f15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:04:01.006202 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd76ebb-66f1-4cb9-806e-14e1a0823f15 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:04:01.009783 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6545e14-ed57-4556-ac04-44a20a7bb1f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:04:01.010400 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6545e14-ed57-4556-ac04-44a20a7bb1f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:04:01.011161 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6545e14-ed57-4556-ac04-44a20a7bb1f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:04:01.011161 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6545e14-ed57-4556-ac04-44a20a7bb1f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:04:01.020958 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6545e14-ed57-4556-ac04-44a20a7bb1f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:04:01.022424 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6545e14-ed57-4556-ac04-44a20a7bb1f4 service neutron] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:04:01.031619 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 584/1186] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:04:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:04:01.481416 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47de26d8-bdbf-40a4-850d-4322ba2ac322 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:04:01.482942 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-47de26d8-bdbf-40a4-850d-4322ba2ac322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:04:01.495490 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47de26d8-bdbf-40a4-850d-4322ba2ac322 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:04:01.500823 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1e24c7-5f7b-4158-af3f-e46877c30a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:04:01.502361 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1e24c7-5f7b-4158-af3f-e46877c30a30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:04:01.507339 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1e24c7-5f7b-4158-af3f-e46877c30a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:04:01.507339 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1e24c7-5f7b-4158-af3f-e46877c30a30 admin admin] PATH_INFO: `/v3/projects/dfdfef40759a48ba80b596f39ced5b47` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:04:01.507339 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1e24c7-5f7b-4158-af3f-e46877c30a30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dfdfef40759a48ba80b596f39ced5b47)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:04:01.507339 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: WARNING py.warnings [None req-6c1e24c7-5f7b-4158-af3f-e46877c30a30 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 14 03:04:01.507339 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: warnings.warn(msg) Oct 14 03:04:01.507339 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: Oct 14 03:04:01.507339 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1e24c7-5f7b-4158-af3f-e46877c30a30 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:04:01.616832 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-6c1e24c7-5f7b-4158-af3f-e46877c30a30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dfdfef40759a48ba80b596f39ced5b47', 'request_id': 'req-6c1e24c7-5f7b-4158-af3f-e46877c30a30'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:04:01.630730 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-6c1e24c7-5f7b-4158-af3f-e46877c30a30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dfdfef40759a48ba80b596f39ced5b47', 'request_id': 'req-6c1e24c7-5f7b-4158-af3f-e46877c30a30'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:04:01.690987 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-6c1e24c7-5f7b-4158-af3f-e46877c30a30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dfdfef40759a48ba80b596f39ced5b47', 'request_id': 'req-6c1e24c7-5f7b-4158-af3f-e46877c30a30'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:04:01.702460 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-6c1e24c7-5f7b-4158-af3f-e46877c30a30 admin admin] The token cache is being invalidate because project dfdfef40759a48ba80b596f39ced5b47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81517) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:04:01.702757 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-6c1e24c7-5f7b-4158-af3f-e46877c30a30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:04:01.703398 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-6c1e24c7-5f7b-4158-af3f-e46877c30a30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:04:01.705006 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 603/1187] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:04:01 2022] DELETE /identity/v3/projects/dfdfef40759a48ba80b596f39ced5b47 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:04:11.027495 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:DELETE=1,keystone:SELECT=37 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:04:11.699106 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:05:27.806338 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f9a2fe-fea9-40c9-a237-ef5552eb2cd4 None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:05:27.806906 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f9a2fe-fea9-40c9-a237-ef5552eb2cd4 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:05:27.806906 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f9a2fe-fea9-40c9-a237-ef5552eb2cd4 None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:05:27.808244 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 585/1188] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 14 03:05:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 03:05:27.813065 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b72411-fa0c-4eb0-8a2b-4e2c50f39b05 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:05:27.813478 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b72411-fa0c-4eb0-8a2b-4e2c50f39b05 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:05:27.813617 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b72411-fa0c-4eb0-8a2b-4e2c50f39b05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:05:27.832984 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-70b72411-fa0c-4eb0-8a2b-4e2c50f39b05 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:05:27.834230 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-70b72411-fa0c-4eb0-8a2b-4e2c50f39b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:05:27.892817 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 604/1189] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 14 03:05:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:05:28.047771 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f9a2fe-fea9-40c9-a237-ef5552eb2cd4 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:05:28.049635 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-24f9a2fe-fea9-40c9-a237-ef5552eb2cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:05:28.151001 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f9a2fe-fea9-40c9-a237-ef5552eb2cd4 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:05:28.157076 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb7c733-4b8d-4de6-ab12-c5fa7d9f1e02 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b07bf0faca47b28b50990d1720853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:05:28.157683 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb7c733-4b8d-4de6-ab12-c5fa7d9f1e02 service nova] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:05:28.157996 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb7c733-4b8d-4de6-ab12-c5fa7d9f1e02 service nova] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:05:28.158273 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb7c733-4b8d-4de6-ab12-c5fa7d9f1e02 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:05:28.166373 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb7c733-4b8d-4de6-ab12-c5fa7d9f1e02 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:05:28.167977 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb7c733-4b8d-4de6-ab12-c5fa7d9f1e02 service nova] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:05:28.177353 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 586/1190] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:05:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:05:29.253960 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78e5769-300f-467c-bf04-e307a5a8f7e3 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:05:29.254530 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78e5769-300f-467c-bf04-e307a5a8f7e3 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:05:29.254530 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78e5769-300f-467c-bf04-e307a5a8f7e3 None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:05:29.255947 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 605/1191] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 14 03:05:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 03:05:29.262175 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cebae12-1fbc-48c4-ad13-46924a96b385 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:05:29.262611 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cebae12-1fbc-48c4-ad13-46924a96b385 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:05:29.262720 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cebae12-1fbc-48c4-ad13-46924a96b385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:05:29.284623 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-3cebae12-1fbc-48c4-ad13-46924a96b385 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:05:29.285915 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-3cebae12-1fbc-48c4-ad13-46924a96b385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:05:29.293816 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 587/1192] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 14 03:05:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:05:29.447416 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78e5769-300f-467c-bf04-e307a5a8f7e3 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:05:29.456730 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e23fcb9-e415-4d38-adf4-4133abccb74f None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:05:29.457017 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e23fcb9-e415-4d38-adf4-4133abccb74f None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:05:29.457300 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e23fcb9-e415-4d38-adf4-4133abccb74f None None] PATH_INFO: `/v3` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:05:29.458589 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 588/1193] 10.222.0.40 () {54 vars in 925 bytes} [Fri Oct 14 03:05:29 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 03:05:29.462681 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78e5769-300f-467c-bf04-e307a5a8f7e3 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:05:29.468790 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4207cf7f-10eb-4ecb-83e7-5ca0fa2efd76 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b07bf0faca47b28b50990d1720853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:05:29.469387 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4207cf7f-10eb-4ecb-83e7-5ca0fa2efd76 service nova] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:05:29.469670 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4207cf7f-10eb-4ecb-83e7-5ca0fa2efd76 service nova] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:05:29.469983 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4207cf7f-10eb-4ecb-83e7-5ca0fa2efd76 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:05:29.475943 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c86c7ab-a51b-41cb-abcb-7ab9cb0ab0f6 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:05:29.476224 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c86c7ab-a51b-41cb-abcb-7ab9cb0ab0f6 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:05:29.476494 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c86c7ab-a51b-41cb-abcb-7ab9cb0ab0f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:05:29.478061 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4207cf7f-10eb-4ecb-83e7-5ca0fa2efd76 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:05:29.479703 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4207cf7f-10eb-4ecb-83e7-5ca0fa2efd76 service nova] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:05:29.489733 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 606/1194] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:05:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:05:29.535850 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-4c86c7ab-a51b-41cb-abcb-7ab9cb0ab0f6 None None] MFA Rules not processed for user `922ef7811aed450baff7f936011b0af1`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:05:29.537091 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-4c86c7ab-a51b-41cb-abcb-7ab9cb0ab0f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:05:29.614357 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 589/1195] 10.222.0.40 () {58 vars in 1015 bytes} [Fri Oct 14 03:05:29 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 03:05:38.557880 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed3a240-960c-4824-9f89-6c48c284d51e None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:05:38.558351 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed3a240-960c-4824-9f89-6c48c284d51e None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:05:38.558461 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed3a240-960c-4824-9f89-6c48c284d51e None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:05:38.559781 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 607/1196] 10.222.0.39 () {54 vars in 961 bytes} [Fri Oct 14 03:05:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 03:05:38.564839 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1e3a10-be90-4e8e-a444-71325e3d40ce None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:05:38.565250 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1e3a10-be90-4e8e-a444-71325e3d40ce None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:05:38.565390 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1e3a10-be90-4e8e-a444-71325e3d40ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:05:38.584744 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-4d1e3a10-be90-4e8e-a444-71325e3d40ce None None] MFA Rules not processed for user `922ef7811aed450baff7f936011b0af1`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:05:38.585978 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-4d1e3a10-be90-4e8e-a444-71325e3d40ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:05:38.595639 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 590/1197] 10.222.0.39 () {58 vars in 1058 bytes} [Fri Oct 14 03:05:38 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:05:38.602932 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed3a240-960c-4824-9f89-6c48c284d51e None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:05:38.616182 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed3a240-960c-4824-9f89-6c48c284d51e None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:05:38.620968 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0b89c0-bbfb-4c8e-843f-02a8245157e6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '922ef7811aed450baff7f936011b0af1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:05:38.621547 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0b89c0-bbfb-4c8e-843f-02a8245157e6 service placement] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:05:38.621876 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0b89c0-bbfb-4c8e-843f-02a8245157e6 service placement] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:05:38.622149 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0b89c0-bbfb-4c8e-843f-02a8245157e6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:05:38.630315 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0b89c0-bbfb-4c8e-843f-02a8245157e6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:05:38.631882 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0b89c0-bbfb-4c8e-843f-02a8245157e6 service placement] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:05:38.641705 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 608/1198] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:05:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 03:05:40.947611 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d7b455-fa78-4ebf-81d3-d977f7deab2b None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:05:40.948127 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d7b455-fa78-4ebf-81d3-d977f7deab2b None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:05:40.948127 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d7b455-fa78-4ebf-81d3-d977f7deab2b None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:05:40.949414 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 591/1199] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 14 03:05:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 03:05:40.956009 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a689479c-a195-4871-bd07-a5a6b7f77bdb None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:05:40.956416 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a689479c-a195-4871-bd07-a5a6b7f77bdb None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:05:40.956589 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a689479c-a195-4871-bd07-a5a6b7f77bdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:05:40.976454 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-a689479c-a195-4871-bd07-a5a6b7f77bdb None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:05:40.977642 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-a689479c-a195-4871-bd07-a5a6b7f77bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:05:40.985633 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 609/1200] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 14 03:05:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:05:41.137746 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d7b455-fa78-4ebf-81d3-d977f7deab2b None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:05:41.152142 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d7b455-fa78-4ebf-81d3-d977f7deab2b None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:05:41.159821 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516dea5b-52a1-49c7-a51b-5781a63419e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b07bf0faca47b28b50990d1720853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:05:41.159821 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516dea5b-52a1-49c7-a51b-5781a63419e1 service nova] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:05:41.159821 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516dea5b-52a1-49c7-a51b-5781a63419e1 service nova] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:05:41.159821 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516dea5b-52a1-49c7-a51b-5781a63419e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:05:41.164657 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:05:41.166972 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516dea5b-52a1-49c7-a51b-5781a63419e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:05:41.168589 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516dea5b-52a1-49c7-a51b-5781a63419e1 service nova] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:05:41.178081 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 592/1201] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:05:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:05:42.251706 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb4e04d-76b5-40a8-b129-8a644346dba2 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:05:42.252389 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb4e04d-76b5-40a8-b129-8a644346dba2 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:05:42.252389 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb4e04d-76b5-40a8-b129-8a644346dba2 None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:05:42.253845 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 610/1202] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 14 03:05:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 03:05:42.260073 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c43de7-bc85-45d2-8828-d96648f16d09 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:05:42.260497 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c43de7-bc85-45d2-8828-d96648f16d09 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:05:42.260627 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c43de7-bc85-45d2-8828-d96648f16d09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:05:42.280979 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-03c43de7-bc85-45d2-8828-d96648f16d09 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:05:42.282162 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-03c43de7-bc85-45d2-8828-d96648f16d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:05:42.289920 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 593/1203] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 14 03:05:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:05:42.440078 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb4e04d-76b5-40a8-b129-8a644346dba2 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:05:42.454580 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb4e04d-76b5-40a8-b129-8a644346dba2 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:05:42.460097 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304746bd-7390-4dff-be1f-6fa1bc84e6a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b07bf0faca47b28b50990d1720853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:05:42.460698 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304746bd-7390-4dff-be1f-6fa1bc84e6a5 service nova] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:05:42.460979 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304746bd-7390-4dff-be1f-6fa1bc84e6a5 service nova] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:05:42.461254 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304746bd-7390-4dff-be1f-6fa1bc84e6a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:05:42.470043 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304746bd-7390-4dff-be1f-6fa1bc84e6a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:05:42.471633 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304746bd-7390-4dff-be1f-6fa1bc84e6a5 service nova] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:05:42.480798 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 611/1204] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:05:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:05:46.455219 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bc1fac-5f0a-44bf-84e3-0558daeeaafd None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:05:46.455801 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bc1fac-5f0a-44bf-84e3-0558daeeaafd None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:05:46.455801 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bc1fac-5f0a-44bf-84e3-0558daeeaafd None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:05:46.457103 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 594/1205] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 14 03:05:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 03:05:46.462252 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9743ec4-3e27-4212-b7c0-5dd9e55cffdd None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:05:46.462672 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9743ec4-3e27-4212-b7c0-5dd9e55cffdd None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:05:46.462805 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9743ec4-3e27-4212-b7c0-5dd9e55cffdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:05:46.482742 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-f9743ec4-3e27-4212-b7c0-5dd9e55cffdd None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:05:46.483993 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-f9743ec4-3e27-4212-b7c0-5dd9e55cffdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:05:46.491895 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 612/1206] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 14 03:05:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:05:46.780636 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bc1fac-5f0a-44bf-84e3-0558daeeaafd None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:05:46.782341 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-a2bc1fac-5f0a-44bf-84e3-0558daeeaafd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:05:46.838217 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bc1fac-5f0a-44bf-84e3-0558daeeaafd None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:05:46.842374 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cc985b-775f-493c-a3ef-436ba7da8b8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:05:46.842976 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cc985b-775f-493c-a3ef-436ba7da8b8e service neutron] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:05:46.843258 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cc985b-775f-493c-a3ef-436ba7da8b8e service neutron] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:05:46.843534 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cc985b-775f-493c-a3ef-436ba7da8b8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:05:46.852261 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cc985b-775f-493c-a3ef-436ba7da8b8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:05:46.853687 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cc985b-775f-493c-a3ef-436ba7da8b8e service neutron] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:05:46.863205 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 595/1207] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:05:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:05:48.989456 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b6a13f-2cd6-4a35-b63d-71b00900ca51 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:05:48.990063 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b6a13f-2cd6-4a35-b63d-71b00900ca51 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:05:48.990063 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b6a13f-2cd6-4a35-b63d-71b00900ca51 None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:05:48.991396 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 613/1208] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 14 03:05:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 03:05:48.996532 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3eb9b3-23f9-49f2-b5af-79b4989a5944 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:05:48.996932 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3eb9b3-23f9-49f2-b5af-79b4989a5944 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:05:48.997089 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3eb9b3-23f9-49f2-b5af-79b4989a5944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:05:49.017076 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-fb3eb9b3-23f9-49f2-b5af-79b4989a5944 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:05:49.018303 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-fb3eb9b3-23f9-49f2-b5af-79b4989a5944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:05:49.026018 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 596/1209] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 14 03:05:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:05:49.313621 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b6a13f-2cd6-4a35-b63d-71b00900ca51 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:05:49.315390 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-97b6a13f-2cd6-4a35-b63d-71b00900ca51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:05:49.328998 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b6a13f-2cd6-4a35-b63d-71b00900ca51 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:05:49.332409 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edf73e2-ba81-42ee-80a3-518289e80b09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:05:49.332996 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf73e2-ba81-42ee-80a3-518289e80b09 service neutron] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:05:49.333274 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf73e2-ba81-42ee-80a3-518289e80b09 service neutron] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:05:49.333550 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf73e2-ba81-42ee-80a3-518289e80b09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:05:49.342790 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edf73e2-ba81-42ee-80a3-518289e80b09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:05:49.344204 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edf73e2-ba81-42ee-80a3-518289e80b09 service neutron] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:05:49.353473 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 614/1210] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:05:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:05:53.003157 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202add75-73ef-493e-b7c5-36512075930e None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:05:53.003728 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202add75-73ef-493e-b7c5-36512075930e None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:05:53.003728 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202add75-73ef-493e-b7c5-36512075930e None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:05:53.005414 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 597/1211] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 14 03:05:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 03:05:53.010985 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f07f50-7aac-4343-9249-c05a5629b7c5 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:05:53.011401 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f07f50-7aac-4343-9249-c05a5629b7c5 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:05:53.011528 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f07f50-7aac-4343-9249-c05a5629b7c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:05:53.019624 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=58 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:05:53.032910 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-07f07f50-7aac-4343-9249-c05a5629b7c5 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:05:53.034144 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-07f07f50-7aac-4343-9249-c05a5629b7c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:05:53.042133 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 615/1212] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 14 03:05:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:05:53.343337 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202add75-73ef-493e-b7c5-36512075930e None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:05:53.344968 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-202add75-73ef-493e-b7c5-36512075930e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:05:53.401071 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202add75-73ef-493e-b7c5-36512075930e None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:05:53.404624 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8445d75e-10bc-423e-b096-4793d8708a0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab599d98b0624a2e89090312bcab713f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:05:53.410366 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445d75e-10bc-423e-b096-4793d8708a0b service cinder] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:05:53.410366 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445d75e-10bc-423e-b096-4793d8708a0b service cinder] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:05:53.410366 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445d75e-10bc-423e-b096-4793d8708a0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:05:53.414323 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8445d75e-10bc-423e-b096-4793d8708a0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:05:53.415744 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8445d75e-10bc-423e-b096-4793d8708a0b service cinder] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:05:53.425425 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 598/1213] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:05:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:05:55.543258 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9073b2b-26f0-4544-b692-b9552ece5228 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:05:55.543854 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9073b2b-26f0-4544-b692-b9552ece5228 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:05:55.543854 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9073b2b-26f0-4544-b692-b9552ece5228 None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:05:55.545154 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 616/1214] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 14 03:05:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 03:05:55.550263 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd33447-4fa1-4e54-91e2-634125a15d98 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:05:55.550661 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd33447-4fa1-4e54-91e2-634125a15d98 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:05:55.550813 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd33447-4fa1-4e54-91e2-634125a15d98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:05:55.570178 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-2fd33447-4fa1-4e54-91e2-634125a15d98 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:05:55.571356 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-2fd33447-4fa1-4e54-91e2-634125a15d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:05:55.579536 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 599/1215] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 14 03:05:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:05:55.882796 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9073b2b-26f0-4544-b692-b9552ece5228 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:05:55.884432 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-b9073b2b-26f0-4544-b692-b9552ece5228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:05:55.897707 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9073b2b-26f0-4544-b692-b9552ece5228 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:05:55.901136 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8478068b-49b2-4dbb-b437-b63e70ffe4d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab599d98b0624a2e89090312bcab713f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:05:55.901722 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8478068b-49b2-4dbb-b437-b63e70ffe4d6 service cinder] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:05:55.902025 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8478068b-49b2-4dbb-b437-b63e70ffe4d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:05:55.902294 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8478068b-49b2-4dbb-b437-b63e70ffe4d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:05:55.911495 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8478068b-49b2-4dbb-b437-b63e70ffe4d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:05:55.912903 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8478068b-49b2-4dbb-b437-b63e70ffe4d6 service cinder] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:05:55.921990 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 617/1216] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:05:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:06:05.564860 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=63 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:06:05.917966 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=8 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:07:20.236747 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad3134a-987e-405f-ba8d-c6c74e84c7ab None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:20.237944 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad3134a-987e-405f-ba8d-c6c74e84c7ab None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:20.238452 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad3134a-987e-405f-ba8d-c6c74e84c7ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:20.262845 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-0ad3134a-987e-405f-ba8d-c6c74e84c7ab None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:20.264403 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-0ad3134a-987e-405f-ba8d-c6c74e84c7ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:20.277255 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 600/1217] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:07:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:07:20.287689 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fddfcef-dff6-4ac1-bb73-f0725147adc5 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:20.302221 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fddfcef-dff6-4ac1-bb73-f0725147adc5 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:20.302221 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fddfcef-dff6-4ac1-bb73-f0725147adc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:20.318785 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-2fddfcef-dff6-4ac1-bb73-f0725147adc5 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:20.320016 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-2fddfcef-dff6-4ac1-bb73-f0725147adc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:20.333239 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 618/1218] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:07:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:07:20.338088 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad3134a-987e-405f-ba8d-c6c74e84c7ab None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:20.351790 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad3134a-987e-405f-ba8d-c6c74e84c7ab None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:20.357499 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f690677-0bcf-416e-b34b-20ac52c98692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:20.358378 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f690677-0bcf-416e-b34b-20ac52c98692 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:20.359265 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f690677-0bcf-416e-b34b-20ac52c98692 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:20.364289 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f690677-0bcf-416e-b34b-20ac52c98692 admin admin] PATH_INFO: `/v3/domains` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:20.365363 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f690677-0bcf-416e-b34b-20ac52c98692 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 03:07:20.365961 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f690677-0bcf-416e-b34b-20ac52c98692 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:20.367210 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: WARNING py.warnings [None req-5f690677-0bcf-416e-b34b-20ac52c98692 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 14 03:07:20.367210 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: warnings.warn(msg) Oct 14 03:07:20.367210 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: Oct 14 03:07:20.368565 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f690677-0bcf-416e-b34b-20ac52c98692 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:20.383122 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 601/1219] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Oct 14 03:07:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 03:07:20.391978 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fddfcef-dff6-4ac1-bb73-f0725147adc5 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:20.405795 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fddfcef-dff6-4ac1-bb73-f0725147adc5 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:20.414145 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72868955-a00c-4ee3-a4be-b4ff66e9d278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:20.414777 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72868955-a00c-4ee3-a4be-b4ff66e9d278 admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:20.415063 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72868955-a00c-4ee3-a4be-b4ff66e9d278 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:20.415343 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72868955-a00c-4ee3-a4be-b4ff66e9d278 admin admin] PATH_INFO: `/v3/projects` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:20.416153 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72868955-a00c-4ee3-a4be-b4ff66e9d278 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:20.417751 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: WARNING py.warnings [None req-72868955-a00c-4ee3-a4be-b4ff66e9d278 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 14 03:07:20.417751 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: warnings.warn(msg) Oct 14 03:07:20.417751 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: Oct 14 03:07:20.418997 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72868955-a00c-4ee3-a4be-b4ff66e9d278 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:20.555085 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 619/1220] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:07:20 2022] POST /identity/v3/projects => generated 373 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:07:20.559443 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f690677-0bcf-416e-b34b-20ac52c98692 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:20.573348 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f690677-0bcf-416e-b34b-20ac52c98692 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:20.614666 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc812a82-cab7-4448-be8b-8501271d59ea None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:20.615023 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc812a82-cab7-4448-be8b-8501271d59ea None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:20.615222 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc812a82-cab7-4448-be8b-8501271d59ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:20.657690 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-fc812a82-cab7-4448-be8b-8501271d59ea None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:20.658980 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-fc812a82-cab7-4448-be8b-8501271d59ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:20.660136 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbc04af-666c-48b7-8024-25768ba9072e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:20.661163 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbc04af-666c-48b7-8024-25768ba9072e admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:20.661884 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbc04af-666c-48b7-8024-25768ba9072e admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:20.662370 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbc04af-666c-48b7-8024-25768ba9072e admin admin] PATH_INFO: `/v3/users` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:20.663269 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbc04af-666c-48b7-8024-25768ba9072e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:20.664529 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: WARNING py.warnings [None req-0fbc04af-666c-48b7-8024-25768ba9072e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 14 03:07:20.664529 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: warnings.warn(msg) Oct 14 03:07:20.664529 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: Oct 14 03:07:20.670775 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbc04af-666c-48b7-8024-25768ba9072e admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:20.686897 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 620/1221] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:07:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Oct 14 03:07:20.696540 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a9f09c-d7e4-42df-92e0-47e2985d9743 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:20.701601 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a9f09c-d7e4-42df-92e0-47e2985d9743 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:20.701601 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a9f09c-d7e4-42df-92e0-47e2985d9743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:20.730995 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-67a9f09c-d7e4-42df-92e0-47e2985d9743 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:20.732715 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-67a9f09c-d7e4-42df-92e0-47e2985d9743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:20.745278 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 621/1222] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:07:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 03:07:20.751236 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a9f09c-d7e4-42df-92e0-47e2985d9743 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:20.764110 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a9f09c-d7e4-42df-92e0-47e2985d9743 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:20.771099 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4ab7c4-7386-4831-be48-d95081100713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:20.772076 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4ab7c4-7386-4831-be48-d95081100713 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:20.772621 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4ab7c4-7386-4831-be48-d95081100713 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:20.773118 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4ab7c4-7386-4831-be48-d95081100713 admin admin] PATH_INFO: `/v3/domains` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:20.774017 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4ab7c4-7386-4831-be48-d95081100713 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 03:07:20.774526 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4ab7c4-7386-4831-be48-d95081100713 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:20.785006 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 602/1223] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:07:20 2022] POST /identity/v3/users => generated 287 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:07:20.786064 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4ab7c4-7386-4831-be48-d95081100713 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:20.793497 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbc04af-666c-48b7-8024-25768ba9072e admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:20.813657 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbc04af-666c-48b7-8024-25768ba9072e admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:20.826836 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 622/1224] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Oct 14 03:07:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 03:07:20.830109 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5990ecff-46d7-45d8-914e-311944784d29 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:20.830785 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ecf531-5769-4cbf-8ecc-a87908b3fd99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:20.831306 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5990ecff-46d7-45d8-914e-311944784d29 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:20.832181 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ecf531-5769-4cbf-8ecc-a87908b3fd99 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:20.832670 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5990ecff-46d7-45d8-914e-311944784d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:20.833491 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ecf531-5769-4cbf-8ecc-a87908b3fd99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:20.836241 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ecf531-5769-4cbf-8ecc-a87908b3fd99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:20.837208 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ecf531-5769-4cbf-8ecc-a87908b3fd99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:20.839651 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ecf531-5769-4cbf-8ecc-a87908b3fd99 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:20.873480 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 603/1225] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:07:20 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:07:20.877588 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ecf531-5769-4cbf-8ecc-a87908b3fd99 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:20.882852 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-5990ecff-46d7-45d8-914e-311944784d29 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:20.886692 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-5990ecff-46d7-45d8-914e-311944784d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:20.898476 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ecf531-5769-4cbf-8ecc-a87908b3fd99 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:20.902921 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 623/1226] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:07:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:07:20.905135 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc80b7-731b-416d-96cc-8f25a578e0bb None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:20.906998 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc80b7-731b-416d-96cc-8f25a578e0bb None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:20.907859 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc80b7-731b-416d-96cc-8f25a578e0bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:20.912566 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f4ea7d-5026-4b45-b66b-d2a5b844bd97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:20.913479 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f4ea7d-5026-4b45-b66b-d2a5b844bd97 admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:20.914005 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f4ea7d-5026-4b45-b66b-d2a5b844bd97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:20.914502 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f4ea7d-5026-4b45-b66b-d2a5b844bd97 admin admin] PATH_INFO: `/v3/projects` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:20.915363 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f4ea7d-5026-4b45-b66b-d2a5b844bd97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:20.920544 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f4ea7d-5026-4b45-b66b-d2a5b844bd97 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:20.944760 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-9fbc80b7-731b-416d-96cc-8f25a578e0bb None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:20.946010 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-9fbc80b7-731b-416d-96cc-8f25a578e0bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:20.954262 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 624/1227] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:07:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:07:20.962313 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38d973c-7f82-4b3b-b872-ba5f6d5ea8fd None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:20.962313 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38d973c-7f82-4b3b-b872-ba5f6d5ea8fd None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:20.962313 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38d973c-7f82-4b3b-b872-ba5f6d5ea8fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:20.980829 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-e38d973c-7f82-4b3b-b872-ba5f6d5ea8fd None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:20.984448 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-e38d973c-7f82-4b3b-b872-ba5f6d5ea8fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:20.990251 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 625/1228] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:07:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:07:20.991043 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38d973c-7f82-4b3b-b872-ba5f6d5ea8fd None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:21.000479 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38d973c-7f82-4b3b-b872-ba5f6d5ea8fd None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:21.008144 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 604/1229] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:07:20 2022] POST /identity/v3/projects => generated 405 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:07:21.009774 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568891cb-d36c-4d75-840e-247de4f7679d None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:21.009774 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568891cb-d36c-4d75-840e-247de4f7679d None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:21.009998 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568891cb-d36c-4d75-840e-247de4f7679d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:21.012662 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c6205e-c119-459c-ac7c-c74c44cc99dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:21.012662 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c6205e-c119-459c-ac7c-c74c44cc99dc admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:21.012662 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c6205e-c119-459c-ac7c-c74c44cc99dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:21.012662 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c6205e-c119-459c-ac7c-c74c44cc99dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:21.013158 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c6205e-c119-459c-ac7c-c74c44cc99dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:21.014748 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c6205e-c119-459c-ac7c-c74c44cc99dc admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:21.028599 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 626/1230] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:07:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:07:21.028599 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ca9bbe-652f-43c4-9bad-1f1d962926af None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:21.028599 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ca9bbe-652f-43c4-9bad-1f1d962926af None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:21.028599 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ca9bbe-652f-43c4-9bad-1f1d962926af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:21.031124 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-568891cb-d36c-4d75-840e-247de4f7679d None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:21.032321 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-568891cb-d36c-4d75-840e-247de4f7679d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:21.046556 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-d0ca9bbe-652f-43c4-9bad-1f1d962926af None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:21.047732 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-d0ca9bbe-652f-43c4-9bad-1f1d962926af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:21.094515 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 605/1231] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:07:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:07:21.096424 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89dd848-12f1-4238-a312-39526c5719fe None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:21.096857 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89dd848-12f1-4238-a312-39526c5719fe None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:21.097137 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89dd848-12f1-4238-a312-39526c5719fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:21.110896 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 627/1232] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:07:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:07:21.118696 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ea84ee-3b85-4f33-b02f-613a2c517af1 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:21.118696 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ea84ee-3b85-4f33-b02f-613a2c517af1 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:21.118696 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ea84ee-3b85-4f33-b02f-613a2c517af1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:21.125642 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-a89dd848-12f1-4238-a312-39526c5719fe None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:21.126927 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-a89dd848-12f1-4238-a312-39526c5719fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:21.131223 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-d2ea84ee-3b85-4f33-b02f-613a2c517af1 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:21.132379 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-d2ea84ee-3b85-4f33-b02f-613a2c517af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:21.134921 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 606/1233] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:07:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:07:21.136634 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28929c2f-9a95-4ebf-b415-4ad0227aa965 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:21.136910 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28929c2f-9a95-4ebf-b415-4ad0227aa965 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:21.137183 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28929c2f-9a95-4ebf-b415-4ad0227aa965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:21.141021 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 628/1234] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:07:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:07:21.142056 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4092796c-f601-4ed8-a160-71ad4a6196ae None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:21.142339 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4092796c-f601-4ed8-a160-71ad4a6196ae None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:21.142610 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4092796c-f601-4ed8-a160-71ad4a6196ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:21.159588 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-28929c2f-9a95-4ebf-b415-4ad0227aa965 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:21.160793 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-28929c2f-9a95-4ebf-b415-4ad0227aa965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:21.162192 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-4092796c-f601-4ed8-a160-71ad4a6196ae None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:21.163333 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-4092796c-f601-4ed8-a160-71ad4a6196ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:21.168966 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 607/1235] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:07:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:07:21.169890 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28929c2f-9a95-4ebf-b415-4ad0227aa965 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:21.171210 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 629/1236] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:07:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:07:21.171814 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4092796c-f601-4ed8-a160-71ad4a6196ae None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:21.181291 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28929c2f-9a95-4ebf-b415-4ad0227aa965 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:21.181712 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4092796c-f601-4ed8-a160-71ad4a6196ae None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:21.186022 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9327d298-250e-4b12-aad7-70abaa6708a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:21.186509 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4b5db2-c3bb-49e6-a7c5-9798c7b8c182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:21.186642 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9327d298-250e-4b12-aad7-70abaa6708a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:21.186879 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9327d298-250e-4b12-aad7-70abaa6708a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:21.187149 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9327d298-250e-4b12-aad7-70abaa6708a2 admin admin] PATH_INFO: `/v3/users` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:21.187258 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b5db2-c3bb-49e6-a7c5-9798c7b8c182 admin admin] REQUEST_METHOD: `PUT` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:21.187506 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b5db2-c3bb-49e6-a7c5-9798c7b8c182 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:21.187774 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9327d298-250e-4b12-aad7-70abaa6708a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:21.187879 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b5db2-c3bb-49e6-a7c5-9798c7b8c182 admin admin] PATH_INFO: `/v3/projects/c48dec42cfcb4cd4a60a97758c3caa4a/users/f06b7e2bb0f8440faf93060e4311468f/roles/9bd0470b544240209a420f640c79c06c` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:21.189688 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9327d298-250e-4b12-aad7-70abaa6708a2 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:21.204193 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4b5db2-c3bb-49e6-a7c5-9798c7b8c182 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c48dec42cfcb4cd4a60a97758c3caa4a, user_id=f06b7e2bb0f8440faf93060e4311468f, role_id=9bd0470b544240209a420f640c79c06c)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:21.207042 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4b5db2-c3bb-49e6-a7c5-9798c7b8c182 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:21.274792 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 608/1237] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:07:21 2022] POST /identity/v3/users => generated 303 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:07:21.275629 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9327d298-250e-4b12-aad7-70abaa6708a2 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:21.285632 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9327d298-250e-4b12-aad7-70abaa6708a2 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:21.289293 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:07:21.292369 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 630/1238] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:07:21 2022] PUT /identity/v3/projects/c48dec42cfcb4cd4a60a97758c3caa4a/users/f06b7e2bb0f8440faf93060e4311468f/roles/9bd0470b544240209a420f640c79c06c => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:07:21.293777 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76358055-4e1a-4612-bf82-8a77749e8a63 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:21.293960 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76358055-4e1a-4612-bf82-8a77749e8a63 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:21.294234 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76358055-4e1a-4612-bf82-8a77749e8a63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:21.313916 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-76358055-4e1a-4612-bf82-8a77749e8a63 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:21.315105 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-76358055-4e1a-4612-bf82-8a77749e8a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:21.342949 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d042546-f65c-4a93-baf7-48f8283827a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:21.343556 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d042546-f65c-4a93-baf7-48f8283827a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:21.343836 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d042546-f65c-4a93-baf7-48f8283827a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:21.344106 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d042546-f65c-4a93-baf7-48f8283827a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:21.344791 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d042546-f65c-4a93-baf7-48f8283827a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 03:07:21.345042 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d042546-f65c-4a93-baf7-48f8283827a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:21.346662 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d042546-f65c-4a93-baf7-48f8283827a8 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:21.349240 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:07:21.359922 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 609/1239] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Oct 14 03:07:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 03:07:21.362848 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33125707-9afb-4ba5-88cb-c6b16359edc9 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:21.362848 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33125707-9afb-4ba5-88cb-c6b16359edc9 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:21.362848 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33125707-9afb-4ba5-88cb-c6b16359edc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:21.376185 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 631/1240] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:07:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:07:21.377566 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8eee8a-079b-4f28-aad1-3bfa62063525 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:21.377889 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8eee8a-079b-4f28-aad1-3bfa62063525 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:21.378183 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8eee8a-079b-4f28-aad1-3bfa62063525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:21.383877 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-33125707-9afb-4ba5-88cb-c6b16359edc9 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:21.385159 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-33125707-9afb-4ba5-88cb-c6b16359edc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:21.394106 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 610/1241] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:07:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:07:21.395045 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33125707-9afb-4ba5-88cb-c6b16359edc9 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:21.398787 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-2f8eee8a-079b-4f28-aad1-3bfa62063525 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:21.399967 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-2f8eee8a-079b-4f28-aad1-3bfa62063525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:21.405105 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33125707-9afb-4ba5-88cb-c6b16359edc9 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:21.408342 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 632/1242] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:07:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:07:21.412000 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8eee8a-079b-4f28-aad1-3bfa62063525 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:21.412126 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0131345e-80ad-4f61-9e85-ab577fe9ee34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:21.412126 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0131345e-80ad-4f61-9e85-ab577fe9ee34 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:21.412126 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0131345e-80ad-4f61-9e85-ab577fe9ee34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:21.412126 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0131345e-80ad-4f61-9e85-ab577fe9ee34 admin admin] PATH_INFO: `/v3/domains` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:21.412524 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0131345e-80ad-4f61-9e85-ab577fe9ee34 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 03:07:21.412775 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0131345e-80ad-4f61-9e85-ab577fe9ee34 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:21.415048 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0131345e-80ad-4f61-9e85-ab577fe9ee34 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:21.418352 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8eee8a-079b-4f28-aad1-3bfa62063525 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:21.423055 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf313fc-85f2-448e-9195-0f78c6684121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:21.423639 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf313fc-85f2-448e-9195-0f78c6684121 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:21.423917 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf313fc-85f2-448e-9195-0f78c6684121 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:21.424187 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf313fc-85f2-448e-9195-0f78c6684121 admin admin] PATH_INFO: `/v3/domains` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:21.424848 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf313fc-85f2-448e-9195-0f78c6684121 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 03:07:21.425150 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf313fc-85f2-448e-9195-0f78c6684121 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:21.426760 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf313fc-85f2-448e-9195-0f78c6684121 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:21.432367 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 611/1243] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Oct 14 03:07:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 03:07:21.432367 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0131345e-80ad-4f61-9e85-ab577fe9ee34 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:21.442716 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0131345e-80ad-4f61-9e85-ab577fe9ee34 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:21.444135 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 633/1244] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Oct 14 03:07:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 03:07:21.452145 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea6fb23-4853-404c-9e4c-d5ab017fcd72 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:21.452145 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea6fb23-4853-404c-9e4c-d5ab017fcd72 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:21.452145 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea6fb23-4853-404c-9e4c-d5ab017fcd72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:21.453956 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1458fba1-6aa3-4f4a-9d50-f5f3f05cd2e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:21.454561 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1458fba1-6aa3-4f4a-9d50-f5f3f05cd2e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:21.454848 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1458fba1-6aa3-4f4a-9d50-f5f3f05cd2e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:21.455131 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1458fba1-6aa3-4f4a-9d50-f5f3f05cd2e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:21.455777 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1458fba1-6aa3-4f4a-9d50-f5f3f05cd2e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:21.457381 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1458fba1-6aa3-4f4a-9d50-f5f3f05cd2e6 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:21.471528 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 612/1245] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:07:21 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:07:21.471528 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1458fba1-6aa3-4f4a-9d50-f5f3f05cd2e6 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:21.477780 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-4ea6fb23-4853-404c-9e4c-d5ab017fcd72 None None] MFA Rules not processed for user `f06b7e2bb0f8440faf93060e4311468f`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:21.479090 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-4ea6fb23-4853-404c-9e4c-d5ab017fcd72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:21.479219 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1458fba1-6aa3-4f4a-9d50-f5f3f05cd2e6 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:21.483687 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e6c00e-55e1-4cc2-8f4f-aa8171964278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:21.486019 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e6c00e-55e1-4cc2-8f4f-aa8171964278 admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:21.486019 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e6c00e-55e1-4cc2-8f4f-aa8171964278 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:21.486019 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e6c00e-55e1-4cc2-8f4f-aa8171964278 admin admin] PATH_INFO: `/v3/projects` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:21.486019 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e6c00e-55e1-4cc2-8f4f-aa8171964278 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:21.487606 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e6c00e-55e1-4cc2-8f4f-aa8171964278 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:21.542092 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 634/1246] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:07:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:07:21.542937 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea6fb23-4853-404c-9e4c-d5ab017fcd72 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:21.552986 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea6fb23-4853-404c-9e4c-d5ab017fcd72 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:21.558056 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93f011b-2e9d-4f5e-86f2-aaa952d7fc43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:21.558650 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93f011b-2e9d-4f5e-86f2-aaa952d7fc43 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:21.558935 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93f011b-2e9d-4f5e-86f2-aaa952d7fc43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:21.559210 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93f011b-2e9d-4f5e-86f2-aaa952d7fc43 admin admin] PATH_INFO: `/v3/domains` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:21.559852 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93f011b-2e9d-4f5e-86f2-aaa952d7fc43 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 03:07:21.560160 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93f011b-2e9d-4f5e-86f2-aaa952d7fc43 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:21.561870 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93f011b-2e9d-4f5e-86f2-aaa952d7fc43 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:21.575218 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 635/1247] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Oct 14 03:07:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 03:07:21.576254 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93f011b-2e9d-4f5e-86f2-aaa952d7fc43 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:21.586153 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93f011b-2e9d-4f5e-86f2-aaa952d7fc43 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:21.587106 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 613/1248] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:07:21 2022] POST /identity/v3/projects => generated 387 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:07:21.587830 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e6c00e-55e1-4cc2-8f4f-aa8171964278 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:21.644107 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e3cd3e-f5ab-4bf7-9dc4-d06359fc3b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:21.644736 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e3cd3e-f5ab-4bf7-9dc4-d06359fc3b4a admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:21.645088 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e3cd3e-f5ab-4bf7-9dc4-d06359fc3b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:21.645434 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e3cd3e-f5ab-4bf7-9dc4-d06359fc3b4a admin admin] PATH_INFO: `/v3/domains` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:21.646112 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e3cd3e-f5ab-4bf7-9dc4-d06359fc3b4a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 03:07:21.646420 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e3cd3e-f5ab-4bf7-9dc4-d06359fc3b4a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:21.648054 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e3cd3e-f5ab-4bf7-9dc4-d06359fc3b4a admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:21.652700 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e6c00e-55e1-4cc2-8f4f-aa8171964278 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:21.658018 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a06a64-d0ff-4d9c-bd5e-51f0dd3ea334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:21.661251 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a06a64-d0ff-4d9c-bd5e-51f0dd3ea334 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:21.661251 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a06a64-d0ff-4d9c-bd5e-51f0dd3ea334 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:21.661251 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a06a64-d0ff-4d9c-bd5e-51f0dd3ea334 admin admin] PATH_INFO: `/v3/domains` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:21.661251 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a06a64-d0ff-4d9c-bd5e-51f0dd3ea334 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 03:07:21.661576 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 636/1249] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Oct 14 03:07:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 03:07:21.661666 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a06a64-d0ff-4d9c-bd5e-51f0dd3ea334 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:21.661743 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e3cd3e-f5ab-4bf7-9dc4-d06359fc3b4a admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:21.661879 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a06a64-d0ff-4d9c-bd5e-51f0dd3ea334 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:21.670686 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e3cd3e-f5ab-4bf7-9dc4-d06359fc3b4a admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:21.675549 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cab7f29-4732-4801-aad0-073ec17a437f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:21.676205 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cab7f29-4732-4801-aad0-073ec17a437f admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:21.676485 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cab7f29-4732-4801-aad0-073ec17a437f admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:21.676759 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cab7f29-4732-4801-aad0-073ec17a437f admin admin] PATH_INFO: `/v3/projects` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:21.677509 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cab7f29-4732-4801-aad0-073ec17a437f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:21.685356 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 614/1250] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Oct 14 03:07:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 03:07:21.685356 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a06a64-d0ff-4d9c-bd5e-51f0dd3ea334 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:21.685596 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cab7f29-4732-4801-aad0-073ec17a437f admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:21.689984 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a06a64-d0ff-4d9c-bd5e-51f0dd3ea334 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:21.694456 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d466cb7-dd01-4ff1-933c-a52fba7a1c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:21.699874 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d466cb7-dd01-4ff1-933c-a52fba7a1c82 admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:21.699874 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d466cb7-dd01-4ff1-933c-a52fba7a1c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:21.699874 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d466cb7-dd01-4ff1-933c-a52fba7a1c82 admin admin] PATH_INFO: `/v3/projects` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:21.699874 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d466cb7-dd01-4ff1-933c-a52fba7a1c82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:21.699874 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d466cb7-dd01-4ff1-933c-a52fba7a1c82 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:21.776008 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 637/1251] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:07:21 2022] POST /identity/v3/projects => generated 377 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:07:21.776008 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cab7f29-4732-4801-aad0-073ec17a437f admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:21.782546 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:07:21.795845 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 615/1252] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:07:21 2022] POST /identity/v3/projects => generated 391 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:07:21.797498 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2162bad5-875d-4875-bbae-1c5282a0e958 None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:21.797817 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2162bad5-875d-4875-bbae-1c5282a0e958 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:21.798117 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2162bad5-875d-4875-bbae-1c5282a0e958 None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:21.799553 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 616/1253] 10.222.0.39 () {54 vars in 961 bytes} [Fri Oct 14 03:07:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 03:07:21.800343 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2162bad5-875d-4875-bbae-1c5282a0e958 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:21.822415 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:07:21.836966 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cab7f29-4732-4801-aad0-073ec17a437f admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:21.866867 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2162bad5-875d-4875-bbae-1c5282a0e958 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:21.872291 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeb5aab-f608-4cd9-815e-1a8df8dc6713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:21.872898 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeb5aab-f608-4cd9-815e-1a8df8dc6713 admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:21.873182 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeb5aab-f608-4cd9-815e-1a8df8dc6713 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:21.873590 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeb5aab-f608-4cd9-815e-1a8df8dc6713 admin admin] PATH_INFO: `/v3/projects` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:21.874293 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeb5aab-f608-4cd9-815e-1a8df8dc6713 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:21.876211 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeb5aab-f608-4cd9-815e-1a8df8dc6713 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:21.894565 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bee7ca-1131-4c88-b033-ce5e8319c79d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:21.895103 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bee7ca-1131-4c88-b033-ce5e8319c79d admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:21.895383 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bee7ca-1131-4c88-b033-ce5e8319c79d admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:21.895656 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bee7ca-1131-4c88-b033-ce5e8319c79d admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:21.896374 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bee7ca-1131-4c88-b033-ce5e8319c79d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:21.898143 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bee7ca-1131-4c88-b033-ce5e8319c79d admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:21.907823 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 638/1254] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:07:21 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:07:21.912185 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bee7ca-1131-4c88-b033-ce5e8319c79d admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:21.918447 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bee7ca-1131-4c88-b033-ce5e8319c79d admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:21.923078 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c4779a-901f-4dc2-8723-5b4d34617422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:21.923685 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c4779a-901f-4dc2-8723-5b4d34617422 admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:21.924007 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c4779a-901f-4dc2-8723-5b4d34617422 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:21.924278 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c4779a-901f-4dc2-8723-5b4d34617422 admin admin] PATH_INFO: `/v3/users` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:21.924884 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c4779a-901f-4dc2-8723-5b4d34617422 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:21.926789 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c4779a-901f-4dc2-8723-5b4d34617422 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:21.973547 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 617/1255] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:07:21 2022] POST /identity/v3/projects => generated 361 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:07:21.974547 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeb5aab-f608-4cd9-815e-1a8df8dc6713 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:22.029052 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 639/1256] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:07:21 2022] POST /identity/v3/users => generated 294 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:07:22.033289 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c4779a-901f-4dc2-8723-5b4d34617422 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:22.038585 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeb5aab-f608-4cd9-815e-1a8df8dc6713 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:22.040790 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe74f6f3-5ddb-4e22-8c13-30c9cf3dd627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:22.041336 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe74f6f3-5ddb-4e22-8c13-30c9cf3dd627 admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:22.042135 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe74f6f3-5ddb-4e22-8c13-30c9cf3dd627 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:22.042423 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe74f6f3-5ddb-4e22-8c13-30c9cf3dd627 admin admin] PATH_INFO: `/v3/projects` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:22.043058 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe74f6f3-5ddb-4e22-8c13-30c9cf3dd627 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:22.043710 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c4779a-901f-4dc2-8723-5b4d34617422 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:22.044990 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe74f6f3-5ddb-4e22-8c13-30c9cf3dd627 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:22.048479 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfe2f6b-30bf-46f8-bff0-6b64bb1d42b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:22.049101 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfe2f6b-30bf-46f8-bff0-6b64bb1d42b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:22.049562 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfe2f6b-30bf-46f8-bff0-6b64bb1d42b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:22.049880 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfe2f6b-30bf-46f8-bff0-6b64bb1d42b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:22.050544 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfe2f6b-30bf-46f8-bff0-6b64bb1d42b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:22.052450 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfe2f6b-30bf-46f8-bff0-6b64bb1d42b3 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:22.124985 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 618/1257] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:07:21 2022] POST /identity/v3/projects => generated 379 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:07:22.124985 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe74f6f3-5ddb-4e22-8c13-30c9cf3dd627 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:22.138916 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 640/1258] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:07:22 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:07:22.140436 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfe2f6b-30bf-46f8-bff0-6b64bb1d42b3 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:22.187655 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe74f6f3-5ddb-4e22-8c13-30c9cf3dd627 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:22.202777 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfe2f6b-30bf-46f8-bff0-6b64bb1d42b3 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:22.207507 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343ec76c-0916-415c-87e6-7330c1c4407d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:22.208130 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343ec76c-0916-415c-87e6-7330c1c4407d admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:22.208453 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343ec76c-0916-415c-87e6-7330c1c4407d admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:22.208764 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343ec76c-0916-415c-87e6-7330c1c4407d admin admin] PATH_INFO: `/v3/users` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:22.209439 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343ec76c-0916-415c-87e6-7330c1c4407d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:22.211507 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343ec76c-0916-415c-87e6-7330c1c4407d admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:22.222021 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:07:22.245845 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffd7499-0b57-477b-98e2-3ec81f3f6448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:22.246459 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffd7499-0b57-477b-98e2-3ec81f3f6448 admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:22.246739 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffd7499-0b57-477b-98e2-3ec81f3f6448 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:22.247010 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffd7499-0b57-477b-98e2-3ec81f3f6448 admin admin] PATH_INFO: `/v3/users` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:22.247630 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffd7499-0b57-477b-98e2-3ec81f3f6448 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:22.249523 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffd7499-0b57-477b-98e2-3ec81f3f6448 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:22.293587 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:07:22.299052 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 641/1259] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:07:22 2022] POST /identity/v3/users => generated 296 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:07:22.300313 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789baa1a-b42b-4984-be55-1827399517ee None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:22.300574 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789baa1a-b42b-4984-be55-1827399517ee None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:22.300861 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789baa1a-b42b-4984-be55-1827399517ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:22.340699 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-789baa1a-b42b-4984-be55-1827399517ee None None] MFA Rules not processed for user `a6b07bf0faca47b28b50990d1720853f`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:22.341945 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-789baa1a-b42b-4984-be55-1827399517ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:22.368795 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 619/1260] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:07:22 2022] POST /identity/v3/users => generated 289 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:07:22.370732 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffd7499-0b57-477b-98e2-3ec81f3f6448 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:22.383080 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffd7499-0b57-477b-98e2-3ec81f3f6448 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:22.390185 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f07480-6982-484d-a688-c705ec54e0a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:22.390871 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f07480-6982-484d-a688-c705ec54e0a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:22.391304 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f07480-6982-484d-a688-c705ec54e0a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:22.391592 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f07480-6982-484d-a688-c705ec54e0a4 admin admin] PATH_INFO: `/v3/projects/b19efffbd45f42158a50d31f0fd97412/users/6e255c4adc56434eb66916220e9ea418/roles/9bd0470b544240209a420f640c79c06c` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:22.409153 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f07480-6982-484d-a688-c705ec54e0a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b19efffbd45f42158a50d31f0fd97412, user_id=6e255c4adc56434eb66916220e9ea418, role_id=9bd0470b544240209a420f640c79c06c)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:22.411844 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 642/1261] 10.222.0.39 () {58 vars in 1058 bytes} [Fri Oct 14 03:07:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:07:22.412197 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f07480-6982-484d-a688-c705ec54e0a4 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:22.412836 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789baa1a-b42b-4984-be55-1827399517ee None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:22.423828 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789baa1a-b42b-4984-be55-1827399517ee None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:22.429113 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50590a1-90ed-435d-a481-3c4cd1739140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:22.430679 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50590a1-90ed-435d-a481-3c4cd1739140 admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:22.430679 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50590a1-90ed-435d-a481-3c4cd1739140 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:22.430679 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50590a1-90ed-435d-a481-3c4cd1739140 admin admin] PATH_INFO: `/v3/users` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:22.431159 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50590a1-90ed-435d-a481-3c4cd1739140 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:22.433279 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50590a1-90ed-435d-a481-3c4cd1739140 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:22.498811 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 620/1262] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:07:22 2022] PUT /identity/v3/projects/b19efffbd45f42158a50d31f0fd97412/users/6e255c4adc56434eb66916220e9ea418/roles/9bd0470b544240209a420f640c79c06c => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:07:22.499680 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f07480-6982-484d-a688-c705ec54e0a4 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:22.522546 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 643/1263] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:07:22 2022] POST /identity/v3/users => generated 281 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:07:22.525111 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50590a1-90ed-435d-a481-3c4cd1739140 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:22.539242 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50590a1-90ed-435d-a481-3c4cd1739140 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:22.572797 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f07480-6982-484d-a688-c705ec54e0a4 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:22.579598 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b289ad00-a565-454f-b49d-73c33833ddc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:22.580216 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b289ad00-a565-454f-b49d-73c33833ddc0 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:22.580497 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b289ad00-a565-454f-b49d-73c33833ddc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:22.580774 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b289ad00-a565-454f-b49d-73c33833ddc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:22.581423 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b289ad00-a565-454f-b49d-73c33833ddc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:22.583077 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b289ad00-a565-454f-b49d-73c33833ddc0 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:22.594380 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 621/1264] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:07:22 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:07:22.595292 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b289ad00-a565-454f-b49d-73c33833ddc0 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:22.600730 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1624a6b8-ae92-4eed-b6cc-d1f8bf905365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:22.601345 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1624a6b8-ae92-4eed-b6cc-d1f8bf905365 admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:22.601627 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1624a6b8-ae92-4eed-b6cc-d1f8bf905365 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:22.601945 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1624a6b8-ae92-4eed-b6cc-d1f8bf905365 admin admin] PATH_INFO: `/v3/users` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:22.602584 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1624a6b8-ae92-4eed-b6cc-d1f8bf905365 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:22.604549 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1624a6b8-ae92-4eed-b6cc-d1f8bf905365 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:22.605085 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b289ad00-a565-454f-b49d-73c33833ddc0 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:22.610061 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dd88b3-fcec-47f7-9dbb-8ca4c1a886de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:22.610658 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dd88b3-fcec-47f7-9dbb-8ca4c1a886de admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:22.610926 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dd88b3-fcec-47f7-9dbb-8ca4c1a886de admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:22.611200 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dd88b3-fcec-47f7-9dbb-8ca4c1a886de admin admin] PATH_INFO: `/v3/users` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:22.611816 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dd88b3-fcec-47f7-9dbb-8ca4c1a886de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:22.613698 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dd88b3-fcec-47f7-9dbb-8ca4c1a886de admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:22.709078 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 644/1265] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:07:22 2022] POST /identity/v3/users => generated 290 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:07:22.709991 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1624a6b8-ae92-4eed-b6cc-d1f8bf905365 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:22.720029 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1624a6b8-ae92-4eed-b6cc-d1f8bf905365 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:22.724725 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca3b06f-caf4-4241-be31-7f8f6cd94288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:22.725302 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca3b06f-caf4-4241-be31-7f8f6cd94288 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:22.725614 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca3b06f-caf4-4241-be31-7f8f6cd94288 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:22.725938 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca3b06f-caf4-4241-be31-7f8f6cd94288 admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:22.726590 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca3b06f-caf4-4241-be31-7f8f6cd94288 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:22.728205 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca3b06f-caf4-4241-be31-7f8f6cd94288 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:22.739013 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 645/1266] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:07:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:07:22.742965 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca3b06f-caf4-4241-be31-7f8f6cd94288 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:22.746971 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 622/1267] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:07:22 2022] POST /identity/v3/users => generated 293 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:07:22.746971 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dd88b3-fcec-47f7-9dbb-8ca4c1a886de admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:22.754282 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca3b06f-caf4-4241-be31-7f8f6cd94288 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:22.758330 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcda531-4c91-4aac-acd9-bda8038c90e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:22.759207 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcda531-4c91-4aac-acd9-bda8038c90e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:22.759486 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcda531-4c91-4aac-acd9-bda8038c90e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:22.759757 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcda531-4c91-4aac-acd9-bda8038c90e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:22.760938 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcda531-4c91-4aac-acd9-bda8038c90e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:22.762561 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcda531-4c91-4aac-acd9-bda8038c90e9 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:22.772572 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 646/1268] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:07:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:07:22.773721 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552af95c-0f0a-48a5-aeb8-bf121a35a48b None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:22.774064 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552af95c-0f0a-48a5-aeb8-bf121a35a48b None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:22.774309 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552af95c-0f0a-48a5-aeb8-bf121a35a48b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:22.793313 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-552af95c-0f0a-48a5-aeb8-bf121a35a48b None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:22.794738 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-552af95c-0f0a-48a5-aeb8-bf121a35a48b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:22.802183 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 647/1269] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:07:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:07:22.802932 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552af95c-0f0a-48a5-aeb8-bf121a35a48b None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:22.812274 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552af95c-0f0a-48a5-aeb8-bf121a35a48b None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:22.816962 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf59a25-07d0-4e49-8724-2913c9bca03f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:22.817548 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf59a25-07d0-4e49-8724-2913c9bca03f admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:22.817856 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf59a25-07d0-4e49-8724-2913c9bca03f admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:22.818173 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf59a25-07d0-4e49-8724-2913c9bca03f admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:22.818815 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf59a25-07d0-4e49-8724-2913c9bca03f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:22.820419 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf59a25-07d0-4e49-8724-2913c9bca03f admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:22.821298 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dd88b3-fcec-47f7-9dbb-8ca4c1a886de admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:22.827627 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a6fc8b-5851-4829-a3d9-bf533049a77e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b07bf0faca47b28b50990d1720853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:22.828241 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a6fc8b-5851-4829-a3d9-bf533049a77e service nova] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:22.828522 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a6fc8b-5851-4829-a3d9-bf533049a77e service nova] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:22.828907 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a6fc8b-5851-4829-a3d9-bf533049a77e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:22.831310 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 648/1270] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:07:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:07:22.832113 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf59a25-07d0-4e49-8724-2913c9bca03f admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:22.844623 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf59a25-07d0-4e49-8724-2913c9bca03f admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:22.847450 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:07:22.850085 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c88259d-04f5-462b-8185-a63589b07cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:22.850664 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c88259d-04f5-462b-8185-a63589b07cd5 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:22.850939 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c88259d-04f5-462b-8185-a63589b07cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:22.851207 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c88259d-04f5-462b-8185-a63589b07cd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:22.851860 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c88259d-04f5-462b-8185-a63589b07cd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:22.853467 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c88259d-04f5-462b-8185-a63589b07cd5 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:22.858607 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:07:22.864304 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 649/1271] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:07:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:07:22.865832 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c88259d-04f5-462b-8185-a63589b07cd5 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:22.874585 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c88259d-04f5-462b-8185-a63589b07cd5 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:22.879162 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bea0262-88f8-4ee7-9385-a8affcdbecfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:22.879766 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bea0262-88f8-4ee7-9385-a8affcdbecfc admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:22.880058 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bea0262-88f8-4ee7-9385-a8affcdbecfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:22.880324 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bea0262-88f8-4ee7-9385-a8affcdbecfc admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:22.880950 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bea0262-88f8-4ee7-9385-a8affcdbecfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:22.882569 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bea0262-88f8-4ee7-9385-a8affcdbecfc admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:22.888199 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a6fc8b-5851-4829-a3d9-bf533049a77e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:22.889842 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a6fc8b-5851-4829-a3d9-bf533049a77e service nova] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:22.893313 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 650/1272] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:07:22 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:07:22.894200 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bea0262-88f8-4ee7-9385-a8affcdbecfc admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:22.898980 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 623/1273] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:07:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:07:22.904955 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a6fc8b-5851-4829-a3d9-bf533049a77e service nova] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:22.905086 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bea0262-88f8-4ee7-9385-a8affcdbecfc admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:22.909746 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ec55e6-f52e-4a23-b79b-9a4ffe7f8a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:22.910143 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a6fc8b-5851-4829-a3d9-bf533049a77e service nova] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:22.910364 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec55e6-f52e-4a23-b79b-9a4ffe7f8a6f admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:22.910645 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec55e6-f52e-4a23-b79b-9a4ffe7f8a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:22.910918 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec55e6-f52e-4a23-b79b-9a4ffe7f8a6f admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:22.911548 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ec55e6-f52e-4a23-b79b-9a4ffe7f8a6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:22.913146 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ec55e6-f52e-4a23-b79b-9a4ffe7f8a6f admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:22.915241 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11f0206-7170-40ef-9529-b3c8b1e32b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:22.915823 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11f0206-7170-40ef-9529-b3c8b1e32b93 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:22.916105 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11f0206-7170-40ef-9529-b3c8b1e32b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:22.916402 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11f0206-7170-40ef-9529-b3c8b1e32b93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:22.917046 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11f0206-7170-40ef-9529-b3c8b1e32b93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:22.918887 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11f0206-7170-40ef-9529-b3c8b1e32b93 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:22.923785 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 651/1274] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:07:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:07:22.924595 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ec55e6-f52e-4a23-b79b-9a4ffe7f8a6f admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:22.931776 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 624/1275] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:07:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:07:22.931776 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11f0206-7170-40ef-9529-b3c8b1e32b93 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:22.938411 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ec55e6-f52e-4a23-b79b-9a4ffe7f8a6f admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:22.943632 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802ba199-4b6d-4be8-8468-bcce3b1a9237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:22.944153 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802ba199-4b6d-4be8-8468-bcce3b1a9237 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:22.944434 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802ba199-4b6d-4be8-8468-bcce3b1a9237 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:22.944709 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802ba199-4b6d-4be8-8468-bcce3b1a9237 admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:22.945383 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802ba199-4b6d-4be8-8468-bcce3b1a9237 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:22.947003 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802ba199-4b6d-4be8-8468-bcce3b1a9237 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:22.958177 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 652/1276] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:07:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:07:22.963458 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802ba199-4b6d-4be8-8468-bcce3b1a9237 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:22.969376 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802ba199-4b6d-4be8-8468-bcce3b1a9237 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:22.977246 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc45722-12b7-408a-bf99-cf823e08734c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:22.977866 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc45722-12b7-408a-bf99-cf823e08734c admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:22.978160 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc45722-12b7-408a-bf99-cf823e08734c admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:22.978420 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc45722-12b7-408a-bf99-cf823e08734c admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:22.979066 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc45722-12b7-408a-bf99-cf823e08734c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:22.980644 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc45722-12b7-408a-bf99-cf823e08734c admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:22.991691 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 653/1277] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:07:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:07:22.992283 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc45722-12b7-408a-bf99-cf823e08734c admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:22.995702 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11f0206-7170-40ef-9529-b3c8b1e32b93 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:22.999548 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade98d45-122f-4185-97b9-fa883bf0a8db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:23.000231 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade98d45-122f-4185-97b9-fa883bf0a8db service neutron] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:23.000571 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade98d45-122f-4185-97b9-fa883bf0a8db service neutron] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:23.000911 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade98d45-122f-4185-97b9-fa883bf0a8db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:23.005528 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc45722-12b7-408a-bf99-cf823e08734c admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:23.014617 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c28c2b-dbce-4932-a9ff-da49b1d20cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:23.015479 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c28c2b-dbce-4932-a9ff-da49b1d20cef admin admin] REQUEST_METHOD: `PUT` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:23.015701 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c28c2b-dbce-4932-a9ff-da49b1d20cef admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:23.016040 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c28c2b-dbce-4932-a9ff-da49b1d20cef admin admin] PATH_INFO: `/v3/projects/55f278e0f09c45ca8ab33cd9e102446c/users/ab344e29f747423fb8766f2d8fbfef0b/roles/9bd0470b544240209a420f640c79c06c` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:23.017883 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade98d45-122f-4185-97b9-fa883bf0a8db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:23.020410 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade98d45-122f-4185-97b9-fa883bf0a8db service neutron] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:23.030406 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 625/1278] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:07:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:07:23.031213 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade98d45-122f-4185-97b9-fa883bf0a8db service neutron] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:23.033475 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c28c2b-dbce-4932-a9ff-da49b1d20cef admin admin] RBAC: Authorizing `identity:create_grant(project_id=55f278e0f09c45ca8ab33cd9e102446c, user_id=ab344e29f747423fb8766f2d8fbfef0b, role_id=9bd0470b544240209a420f640c79c06c)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:23.037696 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c28c2b-dbce-4932-a9ff-da49b1d20cef admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:23.040521 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade98d45-122f-4185-97b9-fa883bf0a8db service neutron] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:23.045383 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f179260a-3d57-4daa-9133-e19e1afd915d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:23.046019 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f179260a-3d57-4daa-9133-e19e1afd915d admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:23.046301 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f179260a-3d57-4daa-9133-e19e1afd915d admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:23.046580 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f179260a-3d57-4daa-9133-e19e1afd915d admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:23.047563 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f179260a-3d57-4daa-9133-e19e1afd915d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:23.048868 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f179260a-3d57-4daa-9133-e19e1afd915d admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:23.059984 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 626/1279] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:07:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:07:23.066259 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f179260a-3d57-4daa-9133-e19e1afd915d admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:23.070541 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f179260a-3d57-4daa-9133-e19e1afd915d admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:23.075785 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f49a36-8f4c-4540-a63b-e10f021ddf4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:23.076391 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f49a36-8f4c-4540-a63b-e10f021ddf4e admin admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:23.076670 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f49a36-8f4c-4540-a63b-e10f021ddf4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:23.076976 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f49a36-8f4c-4540-a63b-e10f021ddf4e admin admin] PATH_INFO: `/v3/projects/44d8a8798bc64aefadd52c30a21002ef/users/7a4ea9e3987e4f0b9ad9ebe6d3c69b7b/roles/9bd0470b544240209a420f640c79c06c` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:23.091632 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f49a36-8f4c-4540-a63b-e10f021ddf4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=44d8a8798bc64aefadd52c30a21002ef, user_id=7a4ea9e3987e4f0b9ad9ebe6d3c69b7b, role_id=9bd0470b544240209a420f640c79c06c)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:23.094543 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f49a36-8f4c-4540-a63b-e10f021ddf4e admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:23.139602 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 654/1280] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:07:22 2022] PUT /identity/v3/projects/55f278e0f09c45ca8ab33cd9e102446c/users/ab344e29f747423fb8766f2d8fbfef0b/roles/9bd0470b544240209a420f640c79c06c => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:07:23.139602 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c28c2b-dbce-4932-a9ff-da49b1d20cef admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:23.157473 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 627/1281] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:07:23 2022] PUT /identity/v3/projects/44d8a8798bc64aefadd52c30a21002ef/users/7a4ea9e3987e4f0b9ad9ebe6d3c69b7b/roles/9bd0470b544240209a420f640c79c06c => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:07:23.158619 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f49a36-8f4c-4540-a63b-e10f021ddf4e admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:23.224305 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c28c2b-dbce-4932-a9ff-da49b1d20cef admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:23.245397 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f49a36-8f4c-4540-a63b-e10f021ddf4e admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:23.251144 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da3af65-ae79-47b6-810b-0e9ee622ce20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:23.251937 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da3af65-ae79-47b6-810b-0e9ee622ce20 admin admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:23.252456 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da3af65-ae79-47b6-810b-0e9ee622ce20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:23.252806 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da3af65-ae79-47b6-810b-0e9ee622ce20 admin admin] PATH_INFO: `/v3/projects/5470d8ee28f74b4eb385a90c33a514b8/users/fb1c279c2f7f404ab9c846a3ba594886/roles/9bd0470b544240209a420f640c79c06c` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:23.270531 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da3af65-ae79-47b6-810b-0e9ee622ce20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5470d8ee28f74b4eb385a90c33a514b8, user_id=fb1c279c2f7f404ab9c846a3ba594886, role_id=9bd0470b544240209a420f640c79c06c)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:23.273347 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da3af65-ae79-47b6-810b-0e9ee622ce20 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:23.290494 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1edaf5-0143-4b49-a07d-e25f289ae7e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:23.291046 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1edaf5-0143-4b49-a07d-e25f289ae7e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:23.291329 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1edaf5-0143-4b49-a07d-e25f289ae7e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:23.291635 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1edaf5-0143-4b49-a07d-e25f289ae7e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:23.292278 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1edaf5-0143-4b49-a07d-e25f289ae7e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:23.293917 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1edaf5-0143-4b49-a07d-e25f289ae7e7 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:23.303859 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 655/1282] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:07:23 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:07:23.304622 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1edaf5-0143-4b49-a07d-e25f289ae7e7 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:23.314518 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1edaf5-0143-4b49-a07d-e25f289ae7e7 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:23.319213 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edf8280-b792-4728-8411-f72e325fa8ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:23.320136 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edf8280-b792-4728-8411-f72e325fa8ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:23.320136 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edf8280-b792-4728-8411-f72e325fa8ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:23.321112 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edf8280-b792-4728-8411-f72e325fa8ac admin admin] PATH_INFO: `/v3/projects/d831dc487bd14af3a83e75aef5bd174d/users/9c5142573a754aa3af69a085560f778b/roles/9bd0470b544240209a420f640c79c06c` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:23.333024 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:07:23.337201 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edf8280-b792-4728-8411-f72e325fa8ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=d831dc487bd14af3a83e75aef5bd174d, user_id=9c5142573a754aa3af69a085560f778b, role_id=9bd0470b544240209a420f640c79c06c)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:23.340040 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edf8280-b792-4728-8411-f72e325fa8ac admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:23.341566 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 628/1283] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:07:23 2022] PUT /identity/v3/projects/5470d8ee28f74b4eb385a90c33a514b8/users/fb1c279c2f7f404ab9c846a3ba594886/roles/9bd0470b544240209a420f640c79c06c => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:07:23.346218 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da3af65-ae79-47b6-810b-0e9ee622ce20 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:23.419977 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 656/1284] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:07:23 2022] PUT /identity/v3/projects/d831dc487bd14af3a83e75aef5bd174d/users/9c5142573a754aa3af69a085560f778b/roles/9bd0470b544240209a420f640c79c06c => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:07:23.421260 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cc9066-9bff-42e7-9080-18f499d04e3c None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:23.421548 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cc9066-9bff-42e7-9080-18f499d04e3c None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:23.421980 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cc9066-9bff-42e7-9080-18f499d04e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:23.449731 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-25cc9066-9bff-42e7-9080-18f499d04e3c None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:23.450985 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-25cc9066-9bff-42e7-9080-18f499d04e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:23.456975 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:07:23.467306 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da3af65-ae79-47b6-810b-0e9ee622ce20 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:23.515815 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 657/1285] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:07:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:07:23.517078 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b289fca-792a-4ac2-8b9b-ee84bcc50a05 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:23.517364 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b289fca-792a-4ac2-8b9b-ee84bcc50a05 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:23.517638 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b289fca-792a-4ac2-8b9b-ee84bcc50a05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:23.532569 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c8c064-fcec-4529-b62b-db88424fe6d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:23.533046 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c8c064-fcec-4529-b62b-db88424fe6d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:23.533427 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c8c064-fcec-4529-b62b-db88424fe6d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:23.533708 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c8c064-fcec-4529-b62b-db88424fe6d9 admin admin] PATH_INFO: `/v3/projects/5cd113c7d65543b2bac8ef368467c092/users/7517af640df3402783a38befbf3d3908/roles/9bd0470b544240209a420f640c79c06c` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:23.537902 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-2b289fca-792a-4ac2-8b9b-ee84bcc50a05 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:23.540689 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-2b289fca-792a-4ac2-8b9b-ee84bcc50a05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:23.547890 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 658/1286] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:07:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:07:23.548887 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b289fca-792a-4ac2-8b9b-ee84bcc50a05 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:23.556223 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c8c064-fcec-4529-b62b-db88424fe6d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cd113c7d65543b2bac8ef368467c092, user_id=7517af640df3402783a38befbf3d3908, role_id=9bd0470b544240209a420f640c79c06c)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:23.559856 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c8c064-fcec-4529-b62b-db88424fe6d9 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:23.565887 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b289fca-792a-4ac2-8b9b-ee84bcc50a05 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:23.570825 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f0d1ca-d93f-42b5-a105-93d72c913a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:23.571507 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f0d1ca-d93f-42b5-a105-93d72c913a01 admin admin] REQUEST_METHOD: `PUT` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:23.571912 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f0d1ca-d93f-42b5-a105-93d72c913a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:23.572186 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f0d1ca-d93f-42b5-a105-93d72c913a01 admin admin] PATH_INFO: `/v3/projects/20fb24be77624e59b8560c7863ca4a46/users/410b7c9e93c94ce78d1b5c0e646307a2/roles/9bd0470b544240209a420f640c79c06c` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:23.588444 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f0d1ca-d93f-42b5-a105-93d72c913a01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20fb24be77624e59b8560c7863ca4a46, user_id=410b7c9e93c94ce78d1b5c0e646307a2, role_id=9bd0470b544240209a420f640c79c06c)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:23.592302 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f0d1ca-d93f-42b5-a105-93d72c913a01 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:23.641655 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 629/1287] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:07:23 2022] PUT /identity/v3/projects/5cd113c7d65543b2bac8ef368467c092/users/7517af640df3402783a38befbf3d3908/roles/9bd0470b544240209a420f640c79c06c => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:07:23.643002 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e9c95a-473e-4107-b4f0-9a26c4ab4963 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:23.643293 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e9c95a-473e-4107-b4f0-9a26c4ab4963 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:23.643706 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e9c95a-473e-4107-b4f0-9a26c4ab4963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:23.663262 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-e6e9c95a-473e-4107-b4f0-9a26c4ab4963 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:23.667488 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-e6e9c95a-473e-4107-b4f0-9a26c4ab4963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:23.673604 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 659/1288] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:07:23 2022] PUT /identity/v3/projects/20fb24be77624e59b8560c7863ca4a46/users/410b7c9e93c94ce78d1b5c0e646307a2/roles/9bd0470b544240209a420f640c79c06c => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:07:23.674415 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae4e78b-6e1f-409e-bbfe-5f93ecb13b4d None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:23.674757 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae4e78b-6e1f-409e-bbfe-5f93ecb13b4d None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:23.675039 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae4e78b-6e1f-409e-bbfe-5f93ecb13b4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:23.711772 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-eae4e78b-6e1f-409e-bbfe-5f93ecb13b4d None None] MFA Rules not processed for user `9c5142573a754aa3af69a085560f778b`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:23.713115 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-eae4e78b-6e1f-409e-bbfe-5f93ecb13b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:23.741730 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 630/1289] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:07:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:07:23.743055 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fe37a4-dc68-471a-87f2-0ce66cb5d9b5 None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:23.743471 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fe37a4-dc68-471a-87f2-0ce66cb5d9b5 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:23.743785 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fe37a4-dc68-471a-87f2-0ce66cb5d9b5 None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:23.745349 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 631/1290] 10.222.0.39 () {54 vars in 961 bytes} [Fri Oct 14 03:07:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 03:07:23.746154 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fe37a4-dc68-471a-87f2-0ce66cb5d9b5 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:23.781387 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:07:23.785320 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 660/1291] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:07:23 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:07:23.786118 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae4e78b-6e1f-409e-bbfe-5f93ecb13b4d None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:23.810442 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fe37a4-dc68-471a-87f2-0ce66cb5d9b5 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:23.814158 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2a9182-0380-4c02-881c-52cd0a983486 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:23.814754 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2a9182-0380-4c02-881c-52cd0a983486 service neutron] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:23.815032 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2a9182-0380-4c02-881c-52cd0a983486 service neutron] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:23.815303 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2a9182-0380-4c02-881c-52cd0a983486 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:23.825431 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2a9182-0380-4c02-881c-52cd0a983486 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:23.829459 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2a9182-0380-4c02-881c-52cd0a983486 service neutron] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:23.837041 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 632/1292] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:07:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:07:23.839871 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfdacf2-1dcf-4449-b5bb-69b1c1f6509f None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:23.840150 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfdacf2-1dcf-4449-b5bb-69b1c1f6509f None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:23.843440 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfdacf2-1dcf-4449-b5bb-69b1c1f6509f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:23.853601 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae4e78b-6e1f-409e-bbfe-5f93ecb13b4d None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:23.860876 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07492bed-ac58-4cb4-922d-0830eb9c3a72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:23.861880 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07492bed-ac58-4cb4-922d-0830eb9c3a72 service neutron] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:23.862376 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07492bed-ac58-4cb4-922d-0830eb9c3a72 service neutron] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:23.862838 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07492bed-ac58-4cb4-922d-0830eb9c3a72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:23.864783 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-0dfdacf2-1dcf-4449-b5bb-69b1c1f6509f None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:23.866092 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-0dfdacf2-1dcf-4449-b5bb-69b1c1f6509f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:23.874371 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07492bed-ac58-4cb4-922d-0830eb9c3a72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:23.875475 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 633/1293] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:07:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:07:23.877527 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3683ce50-6f97-42c3-aa9b-098ecd9e2231 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:23.877953 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3683ce50-6f97-42c3-aa9b-098ecd9e2231 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:23.879364 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07492bed-ac58-4cb4-922d-0830eb9c3a72 service neutron] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:23.882173 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3683ce50-6f97-42c3-aa9b-098ecd9e2231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:23.891037 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 661/1294] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:07:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:07:23.892363 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab22f9dd-750e-4516-90fa-cc918237061c None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:23.892716 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab22f9dd-750e-4516-90fa-cc918237061c None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:23.897507 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab22f9dd-750e-4516-90fa-cc918237061c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:23.906132 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-3683ce50-6f97-42c3-aa9b-098ecd9e2231 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:23.907919 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-3683ce50-6f97-42c3-aa9b-098ecd9e2231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:23.915677 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 634/1295] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:07:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:07:23.917265 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3683ce50-6f97-42c3-aa9b-098ecd9e2231 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:23.919209 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-ab22f9dd-750e-4516-90fa-cc918237061c None None] MFA Rules not processed for user `a6b07bf0faca47b28b50990d1720853f`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:23.921884 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-ab22f9dd-750e-4516-90fa-cc918237061c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:23.929295 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3683ce50-6f97-42c3-aa9b-098ecd9e2231 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:23.934279 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8af0144-1908-4546-94ce-454b69e1b465 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:23.934900 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8af0144-1908-4546-94ce-454b69e1b465 service neutron] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:23.935650 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8af0144-1908-4546-94ce-454b69e1b465 service neutron] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:23.935945 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8af0144-1908-4546-94ce-454b69e1b465 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:23.938088 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:07:23.947600 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8af0144-1908-4546-94ce-454b69e1b465 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:23.954884 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8af0144-1908-4546-94ce-454b69e1b465 service neutron] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:23.960635 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 635/1296] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:07:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:07:23.961628 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8af0144-1908-4546-94ce-454b69e1b465 service neutron] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:23.973718 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8af0144-1908-4546-94ce-454b69e1b465 service neutron] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:23.981217 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fd97a1-6cb8-4267-bcf7-e25d429643d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:23.982125 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fd97a1-6cb8-4267-bcf7-e25d429643d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:23.982757 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fd97a1-6cb8-4267-bcf7-e25d429643d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:23.982757 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fd97a1-6cb8-4267-bcf7-e25d429643d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:23.983516 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fd97a1-6cb8-4267-bcf7-e25d429643d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:23.986488 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fd97a1-6cb8-4267-bcf7-e25d429643d1 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:24.022985 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 662/1297] 10.222.0.39 () {58 vars in 1058 bytes} [Fri Oct 14 03:07:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:07:24.025245 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab22f9dd-750e-4516-90fa-cc918237061c None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:24.037158 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab22f9dd-750e-4516-90fa-cc918237061c None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:24.043640 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bd9fa9-4bc6-4f43-878c-d735ab6b1dec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:24.044256 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bd9fa9-4bc6-4f43-878c-d735ab6b1dec service neutron] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:24.044542 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bd9fa9-4bc6-4f43-878c-d735ab6b1dec service neutron] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:24.044814 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bd9fa9-4bc6-4f43-878c-d735ab6b1dec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:24.054348 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bd9fa9-4bc6-4f43-878c-d735ab6b1dec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:24.055791 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bd9fa9-4bc6-4f43-878c-d735ab6b1dec service neutron] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:24.065329 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 663/1298] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:07:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:07:24.066151 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bd9fa9-4bc6-4f43-878c-d735ab6b1dec service neutron] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:24.077693 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bd9fa9-4bc6-4f43-878c-d735ab6b1dec service neutron] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:24.080181 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331a64cd-0807-4d21-9c21-a6a298e30515 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:24.080689 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331a64cd-0807-4d21-9c21-a6a298e30515 service neutron] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:24.080971 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331a64cd-0807-4d21-9c21-a6a298e30515 service neutron] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:24.081390 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331a64cd-0807-4d21-9c21-a6a298e30515 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:24.091934 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331a64cd-0807-4d21-9c21-a6a298e30515 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:24.093410 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331a64cd-0807-4d21-9c21-a6a298e30515 service neutron] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:24.103526 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 664/1299] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:07:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:07:24.108699 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3881e4-f0b2-4fa1-989c-0523da4027b5 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:24.108699 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3881e4-f0b2-4fa1-989c-0523da4027b5 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:24.108699 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3881e4-f0b2-4fa1-989c-0523da4027b5 None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:24.109232 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 636/1300] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:07:23 2022] POST /identity/v3/projects => generated 365 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:07:24.109331 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 665/1301] 10.222.0.39 () {54 vars in 912 bytes} [Fri Oct 14 03:07:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 03:07:24.110629 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3881e4-f0b2-4fa1-989c-0523da4027b5 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:24.115717 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157f51e7-66c1-4846-8a04-4c515e5ebfe8 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:24.116195 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157f51e7-66c1-4846-8a04-4c515e5ebfe8 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:24.116473 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157f51e7-66c1-4846-8a04-4c515e5ebfe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:24.145434 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-157f51e7-66c1-4846-8a04-4c515e5ebfe8 None None] MFA Rules not processed for user `a6b07bf0faca47b28b50990d1720853f`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:24.146732 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-157f51e7-66c1-4846-8a04-4c515e5ebfe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:24.228377 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3881e4-f0b2-4fa1-989c-0523da4027b5 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:24.233302 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a54788-8eca-4997-80e1-028f24dd938a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:24.234269 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a54788-8eca-4997-80e1-028f24dd938a admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:24.234552 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a54788-8eca-4997-80e1-028f24dd938a admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:24.234818 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a54788-8eca-4997-80e1-028f24dd938a admin admin] PATH_INFO: `/v3/users` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:24.235664 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a54788-8eca-4997-80e1-028f24dd938a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:24.238131 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a54788-8eca-4997-80e1-028f24dd938a admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:24.238550 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 637/1302] 10.222.0.39 () {58 vars in 1009 bytes} [Fri Oct 14 03:07:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:07:24.252274 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157f51e7-66c1-4846-8a04-4c515e5ebfe8 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:24.316356 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157f51e7-66c1-4846-8a04-4c515e5ebfe8 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:24.322113 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315e0692-9bd6-47f9-af26-7059cc73d837 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab599d98b0624a2e89090312bcab713f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:24.325233 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315e0692-9bd6-47f9-af26-7059cc73d837 service cinder] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:24.326199 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315e0692-9bd6-47f9-af26-7059cc73d837 service cinder] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:24.326199 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315e0692-9bd6-47f9-af26-7059cc73d837 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:24.343286 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315e0692-9bd6-47f9-af26-7059cc73d837 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:24.344784 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315e0692-9bd6-47f9-af26-7059cc73d837 service cinder] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:24.357590 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 638/1303] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:07:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 03:07:24.370544 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 666/1304] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:07:24 2022] POST /identity/v3/users => generated 289 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:07:24.371413 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315e0692-9bd6-47f9-af26-7059cc73d837 service cinder] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:24.382363 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315e0692-9bd6-47f9-af26-7059cc73d837 service cinder] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:24.389347 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efc928e-3aca-4b72-aead-a9967271c1fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:24.390105 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc928e-3aca-4b72-aead-a9967271c1fa admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:24.390450 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc928e-3aca-4b72-aead-a9967271c1fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:24.390835 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc928e-3aca-4b72-aead-a9967271c1fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:24.391861 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efc928e-3aca-4b72-aead-a9967271c1fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:24.393598 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efc928e-3aca-4b72-aead-a9967271c1fa admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:24.410667 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 639/1305] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:07:24 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:07:24.411171 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a54788-8eca-4997-80e1-028f24dd938a admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:24.423878 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a54788-8eca-4997-80e1-028f24dd938a admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:24.438571 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f93c5e-018d-40c9-aae7-9aaa391d178e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:24.439899 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f93c5e-018d-40c9-aae7-9aaa391d178e admin admin] REQUEST_METHOD: `PUT` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:24.440228 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f93c5e-018d-40c9-aae7-9aaa391d178e admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:24.440510 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f93c5e-018d-40c9-aae7-9aaa391d178e admin admin] PATH_INFO: `/v3/projects/b34040676bdb4d5f90488f31469fa0f1/users/709bf19619344e3fafd7c04365edb876/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:24.496998 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f93c5e-018d-40c9-aae7-9aaa391d178e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b34040676bdb4d5f90488f31469fa0f1, user_id=709bf19619344e3fafd7c04365edb876, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:24.501341 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f93c5e-018d-40c9-aae7-9aaa391d178e admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:24.544754 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0356dd5a-b988-4236-9e05-540608780f54 None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:24.545320 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0356dd5a-b988-4236-9e05-540608780f54 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:24.545320 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0356dd5a-b988-4236-9e05-540608780f54 None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:24.546709 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 640/1306] 10.222.0.39 () {54 vars in 912 bytes} [Fri Oct 14 03:07:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 14 03:07:24.555619 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 667/1307] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:07:24 2022] PUT /identity/v3/projects/b34040676bdb4d5f90488f31469fa0f1/users/709bf19619344e3fafd7c04365edb876/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 03:07:24.560078 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f93c5e-018d-40c9-aae7-9aaa391d178e admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:24.560708 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eac0da-b232-4593-9f05-bf4966b4c45a None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:24.561153 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eac0da-b232-4593-9f05-bf4966b4c45a None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:24.561600 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eac0da-b232-4593-9f05-bf4966b4c45a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:24.582918 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:07:24.589854 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-a8eac0da-b232-4593-9f05-bf4966b4c45a None None] MFA Rules not processed for user `a6b07bf0faca47b28b50990d1720853f`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:24.591283 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-a8eac0da-b232-4593-9f05-bf4966b4c45a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:24.642634 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:07:24.662895 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f93c5e-018d-40c9-aae7-9aaa391d178e admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:24.669667 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1b36e2-5308-49e9-93b8-34b9333eedc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:24.670512 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1b36e2-5308-49e9-93b8-34b9333eedc3 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:24.670925 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1b36e2-5308-49e9-93b8-34b9333eedc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:24.671388 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1b36e2-5308-49e9-93b8-34b9333eedc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:24.672268 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1b36e2-5308-49e9-93b8-34b9333eedc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:24.675483 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1b36e2-5308-49e9-93b8-34b9333eedc3 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:24.691682 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 668/1308] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:07:24 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:07:24.696196 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1b36e2-5308-49e9-93b8-34b9333eedc3 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:24.719808 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1b36e2-5308-49e9-93b8-34b9333eedc3 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:24.726610 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40c2560-c195-4aaa-88d2-92951833d84f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:24.727636 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40c2560-c195-4aaa-88d2-92951833d84f admin admin] REQUEST_METHOD: `PUT` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:24.728516 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40c2560-c195-4aaa-88d2-92951833d84f admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:24.728516 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40c2560-c195-4aaa-88d2-92951833d84f admin admin] PATH_INFO: `/v3/projects/b34040676bdb4d5f90488f31469fa0f1/users/709bf19619344e3fafd7c04365edb876/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:24.731316 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40c2560-c195-4aaa-88d2-92951833d84f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b34040676bdb4d5f90488f31469fa0f1, user_id=709bf19619344e3fafd7c04365edb876, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:24.734802 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40c2560-c195-4aaa-88d2-92951833d84f admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:24.752949 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 669/1309] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:07:24 2022] PUT /identity/v3/projects/b34040676bdb4d5f90488f31469fa0f1/users/709bf19619344e3fafd7c04365edb876/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:07:24.772573 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa194197-668f-4343-9e0d-3370d898cdfd None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:24.773925 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa194197-668f-4343-9e0d-3370d898cdfd None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:24.774282 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa194197-668f-4343-9e0d-3370d898cdfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:24.806775 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 641/1310] 10.222.0.39 () {58 vars in 1009 bytes} [Fri Oct 14 03:07:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:07:24.823586 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eac0da-b232-4593-9f05-bf4966b4c45a None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:24.831711 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-aa194197-668f-4343-9e0d-3370d898cdfd None None] MFA Rules not processed for user `709bf19619344e3fafd7c04365edb876`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:24.833074 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-aa194197-668f-4343-9e0d-3370d898cdfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:24.919540 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eac0da-b232-4593-9f05-bf4966b4c45a None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:24.991771 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 670/1311] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:07:24 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:07:24.991771 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6a8431-d893-4cb8-80a7-3f26d3530cb8 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:24.991771 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6a8431-d893-4cb8-80a7-3f26d3530cb8 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:24.991771 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6a8431-d893-4cb8-80a7-3f26d3530cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:25.027656 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041d9ecb-5f6e-4247-ba1b-750ca41871ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab599d98b0624a2e89090312bcab713f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:25.030188 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041d9ecb-5f6e-4247-ba1b-750ca41871ae service cinder] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:25.031490 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041d9ecb-5f6e-4247-ba1b-750ca41871ae service cinder] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:25.033455 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041d9ecb-5f6e-4247-ba1b-750ca41871ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:25.051682 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-9d6a8431-d893-4cb8-80a7-3f26d3530cb8 None None] MFA Rules not processed for user `6e255c4adc56434eb66916220e9ea418`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:25.055364 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-9d6a8431-d893-4cb8-80a7-3f26d3530cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:25.143102 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041d9ecb-5f6e-4247-ba1b-750ca41871ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:25.148861 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041d9ecb-5f6e-4247-ba1b-750ca41871ae service cinder] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:25.165678 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 642/1312] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:07:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:07:25.167957 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041d9ecb-5f6e-4247-ba1b-750ca41871ae service cinder] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:25.179644 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041d9ecb-5f6e-4247-ba1b-750ca41871ae service cinder] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:25.185073 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92224460-119b-4d46-b544-66b2011ba995 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab599d98b0624a2e89090312bcab713f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:25.185789 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92224460-119b-4d46-b544-66b2011ba995 service cinder] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:25.186299 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92224460-119b-4d46-b544-66b2011ba995 service cinder] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:25.186772 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92224460-119b-4d46-b544-66b2011ba995 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:25.192376 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 671/1313] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:07:24 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:07:25.193403 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6a8431-d893-4cb8-80a7-3f26d3530cb8 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:25.201664 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92224460-119b-4d46-b544-66b2011ba995 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:25.218905 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92224460-119b-4d46-b544-66b2011ba995 service cinder] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:25.252169 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 643/1314] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:07:25 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:07:25.323937 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6a8431-d893-4cb8-80a7-3f26d3530cb8 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:25.327868 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd24f517-f1b9-414a-b2d4-0cac8cffb9af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:25.328610 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd24f517-f1b9-414a-b2d4-0cac8cffb9af admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:25.328966 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd24f517-f1b9-414a-b2d4-0cac8cffb9af admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:25.329355 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd24f517-f1b9-414a-b2d4-0cac8cffb9af admin admin] PATH_INFO: `/v3/projects` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:25.330256 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd24f517-f1b9-414a-b2d4-0cac8cffb9af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:25.332854 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd24f517-f1b9-414a-b2d4-0cac8cffb9af admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:25.391882 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92224460-119b-4d46-b544-66b2011ba995 service cinder] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:25.405949 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92224460-119b-4d46-b544-66b2011ba995 service cinder] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:25.415917 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdcb448-cad4-43cf-8f9c-89027f6ff72a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:25.417391 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdcb448-cad4-43cf-8f9c-89027f6ff72a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:25.417732 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdcb448-cad4-43cf-8f9c-89027f6ff72a admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:25.419652 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdcb448-cad4-43cf-8f9c-89027f6ff72a admin admin] PATH_INFO: `/v3/users/f06b7e2bb0f8440faf93060e4311468f` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:25.420190 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdcb448-cad4-43cf-8f9c-89027f6ff72a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f06b7e2bb0f8440faf93060e4311468f)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:25.423495 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdcb448-cad4-43cf-8f9c-89027f6ff72a admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:25.457220 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 672/1315] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:07:25 2022] POST /identity/v3/projects => generated 405 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:07:25.458913 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd24f517-f1b9-414a-b2d4-0cac8cffb9af admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:25.509641 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:07:25.575346 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd24f517-f1b9-414a-b2d4-0cac8cffb9af admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:25.591050 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc96e58-96a8-432b-abe3-ee32232662ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:25.592016 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc96e58-96a8-432b-abe3-ee32232662ea admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:25.592016 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc96e58-96a8-432b-abe3-ee32232662ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:25.592985 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc96e58-96a8-432b-abe3-ee32232662ea admin admin] PATH_INFO: `/v3/users` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:25.596670 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc96e58-96a8-432b-abe3-ee32232662ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:25.598943 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc96e58-96a8-432b-abe3-ee32232662ea admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:25.674857 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-ccdcb448-cad4-43cf-8f9c-89027f6ff72a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f06b7e2bb0f8440faf93060e4311468f'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:25.684811 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 673/1316] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:07:25 2022] POST /identity/v3/users => generated 309 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:07:25.685332 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-ccdcb448-cad4-43cf-8f9c-89027f6ff72a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f06b7e2bb0f8440faf93060e4311468f'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:25.685718 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc96e58-96a8-432b-abe3-ee32232662ea admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:25.700708 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-ccdcb448-cad4-43cf-8f9c-89027f6ff72a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f06b7e2bb0f8440faf93060e4311468f'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:25.701927 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-ccdcb448-cad4-43cf-8f9c-89027f6ff72a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f06b7e2bb0f8440faf93060e4311468f'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:25.745105 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc96e58-96a8-432b-abe3-ee32232662ea admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:25.749634 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c4b1b9-c97f-4bb5-8888-e4285e9ab5dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab599d98b0624a2e89090312bcab713f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:25.750204 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-ccdcb448-cad4-43cf-8f9c-89027f6ff72a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f06b7e2bb0f8440faf93060e4311468f'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:25.750446 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c4b1b9-c97f-4bb5-8888-e4285e9ab5dc service cinder] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:25.751135 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c4b1b9-c97f-4bb5-8888-e4285e9ab5dc service cinder] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:25.751417 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c4b1b9-c97f-4bb5-8888-e4285e9ab5dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:25.754740 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:07:25.755739 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-59c4b1b9-c97f-4bb5-8888-e4285e9ab5dc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:25.769259 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 644/1317] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:07:25 2022] DELETE /identity/v3/users/f06b7e2bb0f8440faf93060e4311468f => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 03:07:25.771301 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdcb448-cad4-43cf-8f9c-89027f6ff72a admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:25.774224 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-ccdcb448-cad4-43cf-8f9c-89027f6ff72a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:25.853540 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c4b1b9-c97f-4bb5-8888-e4285e9ab5dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:25.855134 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c4b1b9-c97f-4bb5-8888-e4285e9ab5dc service cinder] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:25.855414 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdcb448-cad4-43cf-8f9c-89027f6ff72a admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:25.865160 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f77c088-5b10-4fc5-b255-2891ec66f94a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:25.865160 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f77c088-5b10-4fc5-b255-2891ec66f94a admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:25.865160 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f77c088-5b10-4fc5-b255-2891ec66f94a admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:25.865160 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f77c088-5b10-4fc5-b255-2891ec66f94a admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:25.865508 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f77c088-5b10-4fc5-b255-2891ec66f94a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:25.867160 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f77c088-5b10-4fc5-b255-2891ec66f94a admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:25.870338 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 674/1318] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:07:25 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:07:25.871738 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29157639-0d18-4b77-8480-e87da34fafc3 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:25.872031 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29157639-0d18-4b77-8480-e87da34fafc3 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:25.872312 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29157639-0d18-4b77-8480-e87da34fafc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:25.895113 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-29157639-0d18-4b77-8480-e87da34fafc3 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:25.896394 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-29157639-0d18-4b77-8480-e87da34fafc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:25.902955 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 645/1319] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:07:25 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:07:25.907644 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f77c088-5b10-4fc5-b255-2891ec66f94a admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:25.908096 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 675/1320] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:07:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:07:25.923006 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f77c088-5b10-4fc5-b255-2891ec66f94a admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:25.928616 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790b0944-12cd-407d-ad36-2f3dc4d9a294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:25.929406 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b0944-12cd-407d-ad36-2f3dc4d9a294 admin admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:25.929849 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b0944-12cd-407d-ad36-2f3dc4d9a294 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:25.930543 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b0944-12cd-407d-ad36-2f3dc4d9a294 admin admin] PATH_INFO: `/v3/projects/c7f6c46584ea4ff19b771019b48c8811/users/80c36f803818485496548836f3a2dc8c/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:25.936145 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29157639-0d18-4b77-8480-e87da34fafc3 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:25.938644 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-29157639-0d18-4b77-8480-e87da34fafc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:25.949533 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790b0944-12cd-407d-ad36-2f3dc4d9a294 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7f6c46584ea4ff19b771019b48c8811, user_id=80c36f803818485496548836f3a2dc8c, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:25.952610 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790b0944-12cd-407d-ad36-2f3dc4d9a294 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:26.048004 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 646/1321] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:07:25 2022] PUT /identity/v3/projects/c7f6c46584ea4ff19b771019b48c8811/users/80c36f803818485496548836f3a2dc8c/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 03:07:26.049731 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29157639-0d18-4b77-8480-e87da34fafc3 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:26.052614 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790b0944-12cd-407d-ad36-2f3dc4d9a294 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:26.134025 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:07:26.167123 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7c5f94-8e5c-49c4-a8f1-02f6e67998b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:26.168046 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7c5f94-8e5c-49c4-a8f1-02f6e67998b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:26.168593 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7c5f94-8e5c-49c4-a8f1-02f6e67998b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:26.174572 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7c5f94-8e5c-49c4-a8f1-02f6e67998b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:26.210618 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790b0944-12cd-407d-ad36-2f3dc4d9a294 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:26.220073 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be838dc-d8bf-4e3f-817a-e1bf68f87e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:26.221087 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be838dc-d8bf-4e3f-817a-e1bf68f87e41 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:26.221661 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be838dc-d8bf-4e3f-817a-e1bf68f87e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:26.222094 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be838dc-d8bf-4e3f-817a-e1bf68f87e41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:26.222766 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be838dc-d8bf-4e3f-817a-e1bf68f87e41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:26.226220 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be838dc-d8bf-4e3f-817a-e1bf68f87e41 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:26.244224 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 647/1322] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:07:26 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 03:07:26.245907 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be838dc-d8bf-4e3f-817a-e1bf68f87e41 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:26.257631 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be838dc-d8bf-4e3f-817a-e1bf68f87e41 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:26.264949 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c764855-7ead-445a-b512-2f5348ab375c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:26.265805 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c764855-7ead-445a-b512-2f5348ab375c admin admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:26.266101 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c764855-7ead-445a-b512-2f5348ab375c admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:26.266383 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c764855-7ead-445a-b512-2f5348ab375c admin admin] PATH_INFO: `/v3/projects/c7f6c46584ea4ff19b771019b48c8811/users/80c36f803818485496548836f3a2dc8c/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:26.279958 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7c5f94-8e5c-49c4-a8f1-02f6e67998b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:26.280969 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c764855-7ead-445a-b512-2f5348ab375c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7f6c46584ea4ff19b771019b48c8811, user_id=80c36f803818485496548836f3a2dc8c, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:26.282320 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7c5f94-8e5c-49c4-a8f1-02f6e67998b8 service neutron] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:26.291500 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c764855-7ead-445a-b512-2f5348ab375c admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:26.296270 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 676/1323] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:07:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 03:07:26.320173 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 648/1324] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:07:26 2022] PUT /identity/v3/projects/c7f6c46584ea4ff19b771019b48c8811/users/80c36f803818485496548836f3a2dc8c/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:07:26.727337 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:07:27.101489 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7c5f94-8e5c-49c4-a8f1-02f6e67998b8 service neutron] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:27.103145 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-db7c5f94-8e5c-49c4-a8f1-02f6e67998b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:27.118440 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7c5f94-8e5c-49c4-a8f1-02f6e67998b8 service neutron] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:27.186385 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba1b8da-8b77-4f46-a330-cf91527fbcbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:27.187403 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba1b8da-8b77-4f46-a330-cf91527fbcbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:27.191451 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba1b8da-8b77-4f46-a330-cf91527fbcbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:27.191451 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba1b8da-8b77-4f46-a330-cf91527fbcbf admin admin] PATH_INFO: `/v3/projects/c48dec42cfcb4cd4a60a97758c3caa4a` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:27.191451 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba1b8da-8b77-4f46-a330-cf91527fbcbf admin admin] RBAC: Authorizing `identity:delete_project(project_id=c48dec42cfcb4cd4a60a97758c3caa4a)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:27.192411 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba1b8da-8b77-4f46-a330-cf91527fbcbf admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:27.355770 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-6ba1b8da-8b77-4f46-a330-cf91527fbcbf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c48dec42cfcb4cd4a60a97758c3caa4a', 'request_id': 'req-6ba1b8da-8b77-4f46-a330-cf91527fbcbf'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:27.366100 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-6ba1b8da-8b77-4f46-a330-cf91527fbcbf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c48dec42cfcb4cd4a60a97758c3caa4a', 'request_id': 'req-6ba1b8da-8b77-4f46-a330-cf91527fbcbf'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:27.447558 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-6ba1b8da-8b77-4f46-a330-cf91527fbcbf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c48dec42cfcb4cd4a60a97758c3caa4a', 'request_id': 'req-6ba1b8da-8b77-4f46-a330-cf91527fbcbf'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:27.489127 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-6ba1b8da-8b77-4f46-a330-cf91527fbcbf admin admin] The token cache is being invalidate because project c48dec42cfcb4cd4a60a97758c3caa4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81517) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:07:27.490525 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-6ba1b8da-8b77-4f46-a330-cf91527fbcbf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:27.492329 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-6ba1b8da-8b77-4f46-a330-cf91527fbcbf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:27.498685 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 677/1325] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:07:27 2022] DELETE /identity/v3/projects/c48dec42cfcb4cd4a60a97758c3caa4a => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:07:27.504089 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0e3d69-b9e6-48a1-ba40-f9725ddad797 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:27.504089 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0e3d69-b9e6-48a1-ba40-f9725ddad797 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:27.504089 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0e3d69-b9e6-48a1-ba40-f9725ddad797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:27.523221 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-2e0e3d69-b9e6-48a1-ba40-f9725ddad797 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:27.524507 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-2e0e3d69-b9e6-48a1-ba40-f9725ddad797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:27.566375 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:07:27.599033 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 649/1326] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:07:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:07:27.607318 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d99a1fc-15fe-44bb-8544-514a3e712ba9 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:27.608131 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d99a1fc-15fe-44bb-8544-514a3e712ba9 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:27.608584 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d99a1fc-15fe-44bb-8544-514a3e712ba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:27.646272 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-6d99a1fc-15fe-44bb-8544-514a3e712ba9 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:27.647711 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-6d99a1fc-15fe-44bb-8544-514a3e712ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:27.661050 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 678/1327] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:07:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:07:27.670830 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0e3d69-b9e6-48a1-ba40-f9725ddad797 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:27.684373 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0e3d69-b9e6-48a1-ba40-f9725ddad797 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:27.699746 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1531d3-dc63-47ae-81ef-ceb87d188ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:27.700481 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1531d3-dc63-47ae-81ef-ceb87d188ed0 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:27.700777 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1531d3-dc63-47ae-81ef-ceb87d188ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:27.701189 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1531d3-dc63-47ae-81ef-ceb87d188ed0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:27.701962 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1531d3-dc63-47ae-81ef-ceb87d188ed0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 03:07:27.702277 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1531d3-dc63-47ae-81ef-ceb87d188ed0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:27.703985 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1531d3-dc63-47ae-81ef-ceb87d188ed0 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:27.725049 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 650/1328] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Oct 14 03:07:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 03:07:27.726674 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d99a1fc-15fe-44bb-8544-514a3e712ba9 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:27.738571 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d99a1fc-15fe-44bb-8544-514a3e712ba9 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:27.743658 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd0f3f8-09eb-4c73-8b00-db630ebfeac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:27.744376 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd0f3f8-09eb-4c73-8b00-db630ebfeac5 admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:27.744693 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd0f3f8-09eb-4c73-8b00-db630ebfeac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:27.744970 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd0f3f8-09eb-4c73-8b00-db630ebfeac5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:27.745845 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd0f3f8-09eb-4c73-8b00-db630ebfeac5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:27.748354 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd0f3f8-09eb-4c73-8b00-db630ebfeac5 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:27.890329 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 679/1329] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:07:27 2022] POST /identity/v3/projects => generated 403 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:07:27.891655 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1531d3-dc63-47ae-81ef-ceb87d188ed0 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:27.906425 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1531d3-dc63-47ae-81ef-ceb87d188ed0 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:27.977426 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8407f9-55bd-4df2-8f9e-ccda1329bd77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:27.978152 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8407f9-55bd-4df2-8f9e-ccda1329bd77 admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:27.978318 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8407f9-55bd-4df2-8f9e-ccda1329bd77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:27.978594 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8407f9-55bd-4df2-8f9e-ccda1329bd77 admin admin] PATH_INFO: `/v3/users` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:27.979266 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8407f9-55bd-4df2-8f9e-ccda1329bd77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:27.981227 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8407f9-55bd-4df2-8f9e-ccda1329bd77 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:28.114421 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 651/1330] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:07:27 2022] POST /identity/v3/users => generated 302 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:07:28.116108 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd0f3f8-09eb-4c73-8b00-db630ebfeac5 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:28.130621 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd0f3f8-09eb-4c73-8b00-db630ebfeac5 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:28.137525 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ad51cd-6ed9-4d2a-90c1-323621b2856d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:28.138762 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ad51cd-6ed9-4d2a-90c1-323621b2856d admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:28.142843 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ad51cd-6ed9-4d2a-90c1-323621b2856d admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:28.142843 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ad51cd-6ed9-4d2a-90c1-323621b2856d admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:28.142843 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ad51cd-6ed9-4d2a-90c1-323621b2856d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:28.143727 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ad51cd-6ed9-4d2a-90c1-323621b2856d admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:28.160239 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 680/1331] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:07:28 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:07:28.162168 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8407f9-55bd-4df2-8f9e-ccda1329bd77 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:28.180487 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8407f9-55bd-4df2-8f9e-ccda1329bd77 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:28.199507 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373ce212-0a8a-4937-9761-ca27d3689397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:28.200160 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373ce212-0a8a-4937-9761-ca27d3689397 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:28.203434 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373ce212-0a8a-4937-9761-ca27d3689397 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:28.203434 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373ce212-0a8a-4937-9761-ca27d3689397 admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:28.203434 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373ce212-0a8a-4937-9761-ca27d3689397 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:28.203434 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373ce212-0a8a-4937-9761-ca27d3689397 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:28.227092 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 652/1332] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:07:28 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:07:28.236911 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ad51cd-6ed9-4d2a-90c1-323621b2856d admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:28.262504 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ad51cd-6ed9-4d2a-90c1-323621b2856d admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:28.275201 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea2f4f0-79cd-4b35-803d-943b274a2bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:28.276112 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea2f4f0-79cd-4b35-803d-943b274a2bd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:28.276467 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea2f4f0-79cd-4b35-803d-943b274a2bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:28.276746 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea2f4f0-79cd-4b35-803d-943b274a2bd4 admin admin] PATH_INFO: `/v3/projects/5ba89a6694304e5ba357c65ae17fecf4/users/5d0b6c544cf84e7fa777b5589974def8/roles/9bd0470b544240209a420f640c79c06c` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:28.303160 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea2f4f0-79cd-4b35-803d-943b274a2bd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ba89a6694304e5ba357c65ae17fecf4, user_id=5d0b6c544cf84e7fa777b5589974def8, role_id=9bd0470b544240209a420f640c79c06c)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:28.306520 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea2f4f0-79cd-4b35-803d-943b274a2bd4 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:28.430125 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 681/1333] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:07:28 2022] PUT /identity/v3/projects/5ba89a6694304e5ba357c65ae17fecf4/users/5d0b6c544cf84e7fa777b5589974def8/roles/9bd0470b544240209a420f640c79c06c => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:07:28.441447 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3d19c0-4193-4458-8f27-f39dfb69396b None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:28.441777 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3d19c0-4193-4458-8f27-f39dfb69396b None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:28.442035 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3d19c0-4193-4458-8f27-f39dfb69396b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:28.502562 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-3f3d19c0-4193-4458-8f27-f39dfb69396b None None] MFA Rules not processed for user `5d0b6c544cf84e7fa777b5589974def8`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:28.503873 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-3f3d19c0-4193-4458-8f27-f39dfb69396b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:28.513963 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:07:28.612596 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 653/1334] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:07:28 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:07:28.614713 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea2f4f0-79cd-4b35-803d-943b274a2bd4 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:28.744714 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea2f4f0-79cd-4b35-803d-943b274a2bd4 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:28.751109 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f85c875-de81-4b0d-b51d-5e83dc8863d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:28.752507 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f85c875-de81-4b0d-b51d-5e83dc8863d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:28.752960 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f85c875-de81-4b0d-b51d-5e83dc8863d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:28.753476 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f85c875-de81-4b0d-b51d-5e83dc8863d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:28.754542 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f85c875-de81-4b0d-b51d-5e83dc8863d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:28.778825 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f85c875-de81-4b0d-b51d-5e83dc8863d2 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:28.929563 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 682/1335] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:07:28 2022] POST /identity/v3/projects => generated 403 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:07:28.930035 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3d19c0-4193-4458-8f27-f39dfb69396b None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:28.950817 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3d19c0-4193-4458-8f27-f39dfb69396b None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:29.042814 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57bca10-2489-4e78-9b25-f6fba5193477 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:29.043154 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57bca10-2489-4e78-9b25-f6fba5193477 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:29.043447 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57bca10-2489-4e78-9b25-f6fba5193477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:29.044292 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5857eb-0fe9-49a5-9b0c-db3ae02320ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:29.045032 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5857eb-0fe9-49a5-9b0c-db3ae02320ef admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:29.045339 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5857eb-0fe9-49a5-9b0c-db3ae02320ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:29.045838 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5857eb-0fe9-49a5-9b0c-db3ae02320ef admin admin] PATH_INFO: `/v3/users` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:29.046569 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5857eb-0fe9-49a5-9b0c-db3ae02320ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:29.048554 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5857eb-0fe9-49a5-9b0c-db3ae02320ef admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:29.085032 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-a57bca10-2489-4e78-9b25-f6fba5193477 None None] MFA Rules not processed for user `80c36f803818485496548836f3a2dc8c`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:29.086977 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-a57bca10-2489-4e78-9b25-f6fba5193477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:29.100335 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:07:29.174089 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 654/1336] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:07:28 2022] POST /identity/v3/users => generated 308 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 03:07:29.179891 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 683/1337] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:07:29 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 03:07:29.184107 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5857eb-0fe9-49a5-9b0c-db3ae02320ef admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:29.199693 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5857eb-0fe9-49a5-9b0c-db3ae02320ef admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:29.206101 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80fb05a-338d-4545-bd17-fc1dca5f002f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:29.207180 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80fb05a-338d-4545-bd17-fc1dca5f002f admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:29.207541 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80fb05a-338d-4545-bd17-fc1dca5f002f admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:29.208203 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80fb05a-338d-4545-bd17-fc1dca5f002f admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:29.208734 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80fb05a-338d-4545-bd17-fc1dca5f002f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:29.211045 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80fb05a-338d-4545-bd17-fc1dca5f002f admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:29.223222 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 655/1338] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:07:29 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 03:07:29.227972 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57bca10-2489-4e78-9b25-f6fba5193477 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:29.246022 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57bca10-2489-4e78-9b25-f6fba5193477 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:29.251920 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f48a2d1-9644-4169-b480-50c1860ab613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:29.252894 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f48a2d1-9644-4169-b480-50c1860ab613 admin admin] REQUEST_METHOD: `PUT` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:29.253382 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f48a2d1-9644-4169-b480-50c1860ab613 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:29.253990 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f48a2d1-9644-4169-b480-50c1860ab613 admin admin] PATH_INFO: `/v3/projects/45877a0d46fb4028aedbe0eb7ac28dc9/users/f02ee2856a014645ab89d827bd9e463f/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:29.298994 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f48a2d1-9644-4169-b480-50c1860ab613 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45877a0d46fb4028aedbe0eb7ac28dc9, user_id=f02ee2856a014645ab89d827bd9e463f, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:29.314494 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f48a2d1-9644-4169-b480-50c1860ab613 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:29.437565 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 684/1339] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:07:29 2022] PUT /identity/v3/projects/45877a0d46fb4028aedbe0eb7ac28dc9/users/f02ee2856a014645ab89d827bd9e463f/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:07:29.441572 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80fb05a-338d-4545-bd17-fc1dca5f002f admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:29.452484 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80fb05a-338d-4545-bd17-fc1dca5f002f admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:29.574757 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2f7e49-19d8-4ef6-ade9-2957d255d6f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:29.576244 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2f7e49-19d8-4ef6-ade9-2957d255d6f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:29.576693 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2f7e49-19d8-4ef6-ade9-2957d255d6f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:29.577188 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2f7e49-19d8-4ef6-ade9-2957d255d6f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:29.578108 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2f7e49-19d8-4ef6-ade9-2957d255d6f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:29.580340 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2f7e49-19d8-4ef6-ade9-2957d255d6f9 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:29.614086 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 656/1340] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:07:29 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:07:29.617114 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f48a2d1-9644-4169-b480-50c1860ab613 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:29.636064 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f48a2d1-9644-4169-b480-50c1860ab613 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:29.643622 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe9572e-5ac6-4a8e-b10a-7521f47bb870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:29.644819 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe9572e-5ac6-4a8e-b10a-7521f47bb870 admin admin] REQUEST_METHOD: `PUT` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:29.645149 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe9572e-5ac6-4a8e-b10a-7521f47bb870 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:29.645408 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe9572e-5ac6-4a8e-b10a-7521f47bb870 admin admin] PATH_INFO: `/v3/projects/45877a0d46fb4028aedbe0eb7ac28dc9/users/f02ee2856a014645ab89d827bd9e463f/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:29.648737 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe9572e-5ac6-4a8e-b10a-7521f47bb870 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45877a0d46fb4028aedbe0eb7ac28dc9, user_id=f02ee2856a014645ab89d827bd9e463f, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:29.658459 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe9572e-5ac6-4a8e-b10a-7521f47bb870 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:29.689189 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 685/1341] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:07:29 2022] PUT /identity/v3/projects/45877a0d46fb4028aedbe0eb7ac28dc9/users/f02ee2856a014645ab89d827bd9e463f/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:07:29.699861 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c7b8c3-f900-491a-84eb-8cb02f74b904 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:29.702973 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c7b8c3-f900-491a-84eb-8cb02f74b904 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:29.702973 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c7b8c3-f900-491a-84eb-8cb02f74b904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:29.755799 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-05c7b8c3-f900-491a-84eb-8cb02f74b904 None None] MFA Rules not processed for user `f02ee2856a014645ab89d827bd9e463f`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:29.757354 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-05c7b8c3-f900-491a-84eb-8cb02f74b904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:29.772101 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:07:30.019441 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 657/1342] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:07:29 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:07:30.055165 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe9572e-5ac6-4a8e-b10a-7521f47bb870 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:30.060444 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-7fe9572e-5ac6-4a8e-b10a-7521f47bb870 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:30.213993 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe9572e-5ac6-4a8e-b10a-7521f47bb870 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:30.224611 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4555943-c42b-494d-b8dd-b8b37339d516 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab599d98b0624a2e89090312bcab713f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:30.232818 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4555943-c42b-494d-b8dd-b8b37339d516 service cinder] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:30.232818 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4555943-c42b-494d-b8dd-b8b37339d516 service cinder] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:30.232818 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4555943-c42b-494d-b8dd-b8b37339d516 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:30.239262 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4555943-c42b-494d-b8dd-b8b37339d516 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:30.240750 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4555943-c42b-494d-b8dd-b8b37339d516 service cinder] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:30.257588 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 686/1343] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:07:30 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:07:30.447162 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c7b8c3-f900-491a-84eb-8cb02f74b904 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:30.447162 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-05c7b8c3-f900-491a-84eb-8cb02f74b904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:30.773090 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4555943-c42b-494d-b8dd-b8b37339d516 service cinder] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:30.788867 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c7b8c3-f900-491a-84eb-8cb02f74b904 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:30.808334 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4555943-c42b-494d-b8dd-b8b37339d516 service cinder] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:30.818320 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b47208e-5ac0-48ec-8845-4c84b4472595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:30.820276 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b47208e-5ac0-48ec-8845-4c84b4472595 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:30.821396 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b47208e-5ac0-48ec-8845-4c84b4472595 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:30.826280 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b47208e-5ac0-48ec-8845-4c84b4472595 admin admin] PATH_INFO: `/v3/users/9c5142573a754aa3af69a085560f778b` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:30.829247 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce78500a-2712-4f09-8de9-9cc524b86fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:30.830141 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b47208e-5ac0-48ec-8845-4c84b4472595 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c5142573a754aa3af69a085560f778b)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:30.831167 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce78500a-2712-4f09-8de9-9cc524b86fad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:30.832754 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce78500a-2712-4f09-8de9-9cc524b86fad admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:30.833120 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce78500a-2712-4f09-8de9-9cc524b86fad admin admin] PATH_INFO: `/v3/users/5d0b6c544cf84e7fa777b5589974def8` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:30.839592 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b47208e-5ac0-48ec-8845-4c84b4472595 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:30.839715 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce78500a-2712-4f09-8de9-9cc524b86fad admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d0b6c544cf84e7fa777b5589974def8)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:30.839715 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: WARNING py.warnings [None req-ce78500a-2712-4f09-8de9-9cc524b86fad admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 14 03:07:30.839715 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: warnings.warn(msg) Oct 14 03:07:30.839715 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: Oct 14 03:07:30.841918 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce78500a-2712-4f09-8de9-9cc524b86fad admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:31.139700 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-4b47208e-5ac0-48ec-8845-4c84b4472595 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c5142573a754aa3af69a085560f778b'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:31.164412 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-4b47208e-5ac0-48ec-8845-4c84b4472595 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c5142573a754aa3af69a085560f778b'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:31.199032 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-ce78500a-2712-4f09-8de9-9cc524b86fad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d0b6c544cf84e7fa777b5589974def8'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:31.202201 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-ce78500a-2712-4f09-8de9-9cc524b86fad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d0b6c544cf84e7fa777b5589974def8'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:31.222035 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-4b47208e-5ac0-48ec-8845-4c84b4472595 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c5142573a754aa3af69a085560f778b'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:31.225906 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-4b47208e-5ac0-48ec-8845-4c84b4472595 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c5142573a754aa3af69a085560f778b'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:31.320715 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-ce78500a-2712-4f09-8de9-9cc524b86fad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d0b6c544cf84e7fa777b5589974def8'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:31.324163 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-ce78500a-2712-4f09-8de9-9cc524b86fad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d0b6c544cf84e7fa777b5589974def8'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:31.334466 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-ce78500a-2712-4f09-8de9-9cc524b86fad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d0b6c544cf84e7fa777b5589974def8'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:31.347756 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:07:31.362677 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 687/1344] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:07:30 2022] DELETE /identity/v3/users/5d0b6c544cf84e7fa777b5589974def8 => generated 0 bytes in 591 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:07:31.364416 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4071f4d-6d99-4054-8cbe-8375f65bcb98 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:31.364969 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4071f4d-6d99-4054-8cbe-8375f65bcb98 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:31.365712 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4071f4d-6d99-4054-8cbe-8375f65bcb98 None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:31.367306 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 688/1345] 10.222.0.39 () {54 vars in 921 bytes} [Fri Oct 14 03:07:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 03:07:31.368685 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeddaa8-55fc-4fb3-976f-9a9e4f077691 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:31.369030 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeddaa8-55fc-4fb3-976f-9a9e4f077691 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:31.369399 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeddaa8-55fc-4fb3-976f-9a9e4f077691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:31.388767 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-6aeddaa8-55fc-4fb3-976f-9a9e4f077691 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:31.390150 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-6aeddaa8-55fc-4fb3-976f-9a9e4f077691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:31.394851 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-4b47208e-5ac0-48ec-8845-4c84b4472595 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c5142573a754aa3af69a085560f778b'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:31.405657 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 658/1346] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:07:30 2022] DELETE /identity/v3/users/9c5142573a754aa3af69a085560f778b => generated 0 bytes in 959 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:07:31.407100 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c63ac04-1fe8-4cc5-9c67-a291b11a57c6 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:31.407759 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c63ac04-1fe8-4cc5-9c67-a291b11a57c6 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:31.407960 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c63ac04-1fe8-4cc5-9c67-a291b11a57c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:31.432797 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-0c63ac04-1fe8-4cc5-9c67-a291b11a57c6 None None] MFA Rules not processed for user `922ef7811aed450baff7f936011b0af1`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:31.435089 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-0c63ac04-1fe8-4cc5-9c67-a291b11a57c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:31.455066 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:07:31.504580 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 689/1347] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:07:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:07:31.514959 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c238b401-823f-41d6-a946-aaa1bb867e6f None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:31.515249 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c238b401-823f-41d6-a946-aaa1bb867e6f None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:31.519702 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c238b401-823f-41d6-a946-aaa1bb867e6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:31.528456 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 659/1348] 10.222.0.39 () {58 vars in 1018 bytes} [Fri Oct 14 03:07:31 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:07:31.529645 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c63ac04-1fe8-4cc5-9c67-a291b11a57c6 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:31.531295 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-0c63ac04-1fe8-4cc5-9c67-a291b11a57c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:31.548427 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-c238b401-823f-41d6-a946-aaa1bb867e6f None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:31.551261 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-c238b401-823f-41d6-a946-aaa1bb867e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:31.618632 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c63ac04-1fe8-4cc5-9c67-a291b11a57c6 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:31.637471 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8690c699-9489-4b7e-9b36-10b3c8f26710 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:31.638145 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8690c699-9489-4b7e-9b36-10b3c8f26710 service neutron] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:31.638425 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8690c699-9489-4b7e-9b36-10b3c8f26710 service neutron] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:31.638693 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8690c699-9489-4b7e-9b36-10b3c8f26710 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:31.670220 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 690/1349] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:07:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:07:31.671257 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c238b401-823f-41d6-a946-aaa1bb867e6f None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:31.681816 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-c238b401-823f-41d6-a946-aaa1bb867e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:31.695855 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c238b401-823f-41d6-a946-aaa1bb867e6f None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:31.705170 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c59ba1-f641-4e0e-afd9-a5d783b8b0db service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '922ef7811aed450baff7f936011b0af1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:31.705901 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c59ba1-f641-4e0e-afd9-a5d783b8b0db service placement] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:31.706190 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c59ba1-f641-4e0e-afd9-a5d783b8b0db service placement] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:31.706466 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c59ba1-f641-4e0e-afd9-a5d783b8b0db service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:31.727078 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c59ba1-f641-4e0e-afd9-a5d783b8b0db service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:31.730789 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c59ba1-f641-4e0e-afd9-a5d783b8b0db service placement] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:31.751389 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 691/1350] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:07:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:07:31.752708 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c59ba1-f641-4e0e-afd9-a5d783b8b0db service placement] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:31.769629 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c59ba1-f641-4e0e-afd9-a5d783b8b0db service placement] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:31.798689 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7166b3a7-e73e-40a0-8d4f-52b3e9827d9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:31.801787 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7166b3a7-e73e-40a0-8d4f-52b3e9827d9f service neutron] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:31.802909 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7166b3a7-e73e-40a0-8d4f-52b3e9827d9f service neutron] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:31.804353 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7166b3a7-e73e-40a0-8d4f-52b3e9827d9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:31.820718 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8690c699-9489-4b7e-9b36-10b3c8f26710 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:31.823468 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8690c699-9489-4b7e-9b36-10b3c8f26710 service neutron] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:31.850169 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7166b3a7-e73e-40a0-8d4f-52b3e9827d9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:31.853679 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7166b3a7-e73e-40a0-8d4f-52b3e9827d9f service neutron] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:31.888866 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 660/1351] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:07:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:07:31.911573 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 692/1352] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:07:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:07:33.358127 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8690c699-9489-4b7e-9b36-10b3c8f26710 service neutron] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:33.360493 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-8690c699-9489-4b7e-9b36-10b3c8f26710 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:33.394430 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8690c699-9489-4b7e-9b36-10b3c8f26710 service neutron] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:33.402335 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d8f040-98f9-4bf2-85be-f94bc0ee6d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:33.405506 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d8f040-98f9-4bf2-85be-f94bc0ee6d68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:33.405506 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d8f040-98f9-4bf2-85be-f94bc0ee6d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:33.405927 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d8f040-98f9-4bf2-85be-f94bc0ee6d68 admin admin] PATH_INFO: `/v3/projects/d831dc487bd14af3a83e75aef5bd174d` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:33.408223 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d8f040-98f9-4bf2-85be-f94bc0ee6d68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d831dc487bd14af3a83e75aef5bd174d)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:33.409373 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: WARNING py.warnings [None req-99d8f040-98f9-4bf2-85be-f94bc0ee6d68 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 14 03:07:33.409373 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: warnings.warn(msg) Oct 14 03:07:33.409373 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: Oct 14 03:07:33.410780 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d8f040-98f9-4bf2-85be-f94bc0ee6d68 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:33.465232 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7166b3a7-e73e-40a0-8d4f-52b3e9827d9f service neutron] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:33.468109 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-7166b3a7-e73e-40a0-8d4f-52b3e9827d9f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:33.497582 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7166b3a7-e73e-40a0-8d4f-52b3e9827d9f service neutron] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:33.513198 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7603f693-e7ac-4f3c-9224-c553b310e3af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:33.513829 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7603f693-e7ac-4f3c-9224-c553b310e3af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:33.514103 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7603f693-e7ac-4f3c-9224-c553b310e3af admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:33.514378 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7603f693-e7ac-4f3c-9224-c553b310e3af admin admin] PATH_INFO: `/v3/users/6e255c4adc56434eb66916220e9ea418` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:33.516848 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7603f693-e7ac-4f3c-9224-c553b310e3af admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e255c4adc56434eb66916220e9ea418)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:33.523809 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7603f693-e7ac-4f3c-9224-c553b310e3af admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:33.591315 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-99d8f040-98f9-4bf2-85be-f94bc0ee6d68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd831dc487bd14af3a83e75aef5bd174d', 'request_id': 'req-99d8f040-98f9-4bf2-85be-f94bc0ee6d68'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:33.611237 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-99d8f040-98f9-4bf2-85be-f94bc0ee6d68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd831dc487bd14af3a83e75aef5bd174d', 'request_id': 'req-99d8f040-98f9-4bf2-85be-f94bc0ee6d68'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:33.720823 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-99d8f040-98f9-4bf2-85be-f94bc0ee6d68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd831dc487bd14af3a83e75aef5bd174d', 'request_id': 'req-99d8f040-98f9-4bf2-85be-f94bc0ee6d68'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:33.728668 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:07:33.738794 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-99d8f040-98f9-4bf2-85be-f94bc0ee6d68 admin admin] The token cache is being invalidate because project d831dc487bd14af3a83e75aef5bd174d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:07:33.739849 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-99d8f040-98f9-4bf2-85be-f94bc0ee6d68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:33.742221 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-99d8f040-98f9-4bf2-85be-f94bc0ee6d68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:33.746080 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 661/1353] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:07:33 2022] DELETE /identity/v3/projects/d831dc487bd14af3a83e75aef5bd174d => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:07:33.748266 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d8f040-98f9-4bf2-85be-f94bc0ee6d68 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:33.751476 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-99d8f040-98f9-4bf2-85be-f94bc0ee6d68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:33.797143 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-7603f693-e7ac-4f3c-9224-c553b310e3af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e255c4adc56434eb66916220e9ea418'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:33.799202 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-7603f693-e7ac-4f3c-9224-c553b310e3af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e255c4adc56434eb66916220e9ea418'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:33.875637 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d8f040-98f9-4bf2-85be-f94bc0ee6d68 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:33.881453 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5fb327-cb04-4e8f-a701-b9f9d39cb372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:33.882203 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5fb327-cb04-4e8f-a701-b9f9d39cb372 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:33.882490 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5fb327-cb04-4e8f-a701-b9f9d39cb372 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:33.882910 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5fb327-cb04-4e8f-a701-b9f9d39cb372 admin admin] PATH_INFO: `/v3/projects/5ba89a6694304e5ba357c65ae17fecf4` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:33.884157 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5fb327-cb04-4e8f-a701-b9f9d39cb372 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ba89a6694304e5ba357c65ae17fecf4)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:33.886423 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5fb327-cb04-4e8f-a701-b9f9d39cb372 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:33.892991 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-7603f693-e7ac-4f3c-9224-c553b310e3af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e255c4adc56434eb66916220e9ea418'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:33.896633 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-7603f693-e7ac-4f3c-9224-c553b310e3af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e255c4adc56434eb66916220e9ea418'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:33.896633 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:07:33.910514 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-7603f693-e7ac-4f3c-9224-c553b310e3af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e255c4adc56434eb66916220e9ea418'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:33.934062 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 693/1354] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:07:33 2022] DELETE /identity/v3/users/6e255c4adc56434eb66916220e9ea418 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 03:07:33.934682 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7603f693-e7ac-4f3c-9224-c553b310e3af admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:33.936093 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-7603f693-e7ac-4f3c-9224-c553b310e3af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:34.026604 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7603f693-e7ac-4f3c-9224-c553b310e3af admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:34.042367 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-3a5fb327-cb04-4e8f-a701-b9f9d39cb372 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ba89a6694304e5ba357c65ae17fecf4', 'request_id': 'req-3a5fb327-cb04-4e8f-a701-b9f9d39cb372'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:34.053562 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-3a5fb327-cb04-4e8f-a701-b9f9d39cb372 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ba89a6694304e5ba357c65ae17fecf4', 'request_id': 'req-3a5fb327-cb04-4e8f-a701-b9f9d39cb372'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:34.115273 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505d72ec-55f6-4da7-aace-9b5bfe0da325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:34.116023 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505d72ec-55f6-4da7-aace-9b5bfe0da325 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:34.116530 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505d72ec-55f6-4da7-aace-9b5bfe0da325 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:34.116684 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505d72ec-55f6-4da7-aace-9b5bfe0da325 admin admin] PATH_INFO: `/v3/users/709bf19619344e3fafd7c04365edb876` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:34.118377 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505d72ec-55f6-4da7-aace-9b5bfe0da325 admin admin] RBAC: Authorizing `identity:delete_user(user_id=709bf19619344e3fafd7c04365edb876)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:34.122414 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505d72ec-55f6-4da7-aace-9b5bfe0da325 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:34.161346 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-3a5fb327-cb04-4e8f-a701-b9f9d39cb372 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ba89a6694304e5ba357c65ae17fecf4', 'request_id': 'req-3a5fb327-cb04-4e8f-a701-b9f9d39cb372'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:34.177434 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-3a5fb327-cb04-4e8f-a701-b9f9d39cb372 admin admin] The token cache is being invalidate because project 5ba89a6694304e5ba357c65ae17fecf4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:07:34.177434 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-3a5fb327-cb04-4e8f-a701-b9f9d39cb372 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:34.178443 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-3a5fb327-cb04-4e8f-a701-b9f9d39cb372 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:34.182793 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 662/1355] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:07:33 2022] DELETE /identity/v3/projects/5ba89a6694304e5ba357c65ae17fecf4 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:07:34.186410 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5fb327-cb04-4e8f-a701-b9f9d39cb372 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:34.188092 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-3a5fb327-cb04-4e8f-a701-b9f9d39cb372 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:34.202641 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5fb327-cb04-4e8f-a701-b9f9d39cb372 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:34.208309 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e24cc6-417f-43ae-8c14-e66918918089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:34.209397 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e24cc6-417f-43ae-8c14-e66918918089 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:34.209397 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e24cc6-417f-43ae-8c14-e66918918089 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:34.209778 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e24cc6-417f-43ae-8c14-e66918918089 admin admin] PATH_INFO: `/v3/users/f02ee2856a014645ab89d827bd9e463f` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:34.211259 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e24cc6-417f-43ae-8c14-e66918918089 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f02ee2856a014645ab89d827bd9e463f)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:34.214327 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e24cc6-417f-43ae-8c14-e66918918089 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:34.385331 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-505d72ec-55f6-4da7-aace-9b5bfe0da325 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '709bf19619344e3fafd7c04365edb876'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:34.386078 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-505d72ec-55f6-4da7-aace-9b5bfe0da325 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '709bf19619344e3fafd7c04365edb876'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:34.386304 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-15e24cc6-417f-43ae-8c14-e66918918089 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f02ee2856a014645ab89d827bd9e463f'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:34.397896 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-15e24cc6-417f-43ae-8c14-e66918918089 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f02ee2856a014645ab89d827bd9e463f'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:34.417081 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-15e24cc6-417f-43ae-8c14-e66918918089 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f02ee2856a014645ab89d827bd9e463f'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:34.419027 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-15e24cc6-417f-43ae-8c14-e66918918089 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f02ee2856a014645ab89d827bd9e463f'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:34.476045 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-505d72ec-55f6-4da7-aace-9b5bfe0da325 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '709bf19619344e3fafd7c04365edb876'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:34.477160 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-505d72ec-55f6-4da7-aace-9b5bfe0da325 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '709bf19619344e3fafd7c04365edb876'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:34.490101 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-505d72ec-55f6-4da7-aace-9b5bfe0da325 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '709bf19619344e3fafd7c04365edb876'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:34.526794 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 694/1356] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:07:33 2022] DELETE /identity/v3/users/709bf19619344e3fafd7c04365edb876 => generated 0 bytes in 592 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:07:34.527507 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505d72ec-55f6-4da7-aace-9b5bfe0da325 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:34.530845 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-505d72ec-55f6-4da7-aace-9b5bfe0da325 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:34.586201 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-15e24cc6-417f-43ae-8c14-e66918918089 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f02ee2856a014645ab89d827bd9e463f'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:34.612611 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 663/1357] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:07:34 2022] DELETE /identity/v3/users/f02ee2856a014645ab89d827bd9e463f => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:07:34.691929 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505d72ec-55f6-4da7-aace-9b5bfe0da325 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:34.790394 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdcf466-a08f-4fa8-bc5f-118824e9fa64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:34.791993 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdcf466-a08f-4fa8-bc5f-118824e9fa64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:34.792780 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdcf466-a08f-4fa8-bc5f-118824e9fa64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:34.793137 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdcf466-a08f-4fa8-bc5f-118824e9fa64 admin admin] PATH_INFO: `/v3/projects/b19efffbd45f42158a50d31f0fd97412` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:34.795146 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdcf466-a08f-4fa8-bc5f-118824e9fa64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b19efffbd45f42158a50d31f0fd97412)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:34.798356 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdcf466-a08f-4fa8-bc5f-118824e9fa64 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:34.981870 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-efdcf466-a08f-4fa8-bc5f-118824e9fa64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b19efffbd45f42158a50d31f0fd97412', 'request_id': 'req-efdcf466-a08f-4fa8-bc5f-118824e9fa64'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:34.996117 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-efdcf466-a08f-4fa8-bc5f-118824e9fa64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b19efffbd45f42158a50d31f0fd97412', 'request_id': 'req-efdcf466-a08f-4fa8-bc5f-118824e9fa64'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:35.069075 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-efdcf466-a08f-4fa8-bc5f-118824e9fa64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b19efffbd45f42158a50d31f0fd97412', 'request_id': 'req-efdcf466-a08f-4fa8-bc5f-118824e9fa64'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:35.082400 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:07:35.091018 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-efdcf466-a08f-4fa8-bc5f-118824e9fa64 admin admin] The token cache is being invalidate because project b19efffbd45f42158a50d31f0fd97412 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81517) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:07:35.091869 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-efdcf466-a08f-4fa8-bc5f-118824e9fa64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:35.092304 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-efdcf466-a08f-4fa8-bc5f-118824e9fa64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:35.098541 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e24cc6-417f-43ae-8c14-e66918918089 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:35.098917 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 695/1358] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:07:34 2022] DELETE /identity/v3/projects/b19efffbd45f42158a50d31f0fd97412 => generated 0 bytes in 567 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:07:35.101303 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-15e24cc6-417f-43ae-8c14-e66918918089 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:35.140736 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e24cc6-417f-43ae-8c14-e66918918089 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:35.167496 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdcf466-a08f-4fa8-bc5f-118824e9fa64 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:35.169346 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-efdcf466-a08f-4fa8-bc5f-118824e9fa64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:35.257273 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7002ae8-d374-433f-8e07-6c85700fcbb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:35.258549 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7002ae8-d374-433f-8e07-6c85700fcbb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:35.259087 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7002ae8-d374-433f-8e07-6c85700fcbb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:35.259599 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7002ae8-d374-433f-8e07-6c85700fcbb2 admin admin] PATH_INFO: `/v3/users/80c36f803818485496548836f3a2dc8c` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:35.264499 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7002ae8-d374-433f-8e07-6c85700fcbb2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80c36f803818485496548836f3a2dc8c)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:35.269112 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7002ae8-d374-433f-8e07-6c85700fcbb2 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:35.348691 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdcf466-a08f-4fa8-bc5f-118824e9fa64 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:35.356940 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9540fdbb-311e-4edd-a125-22c4fd124102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:35.357793 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9540fdbb-311e-4edd-a125-22c4fd124102 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:35.358170 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9540fdbb-311e-4edd-a125-22c4fd124102 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:35.358581 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9540fdbb-311e-4edd-a125-22c4fd124102 admin admin] PATH_INFO: `/v3/projects/b34040676bdb4d5f90488f31469fa0f1` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:35.360387 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9540fdbb-311e-4edd-a125-22c4fd124102 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b34040676bdb4d5f90488f31469fa0f1)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:35.364861 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9540fdbb-311e-4edd-a125-22c4fd124102 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:35.458084 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:07:35.502449 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-d7002ae8-d374-433f-8e07-6c85700fcbb2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80c36f803818485496548836f3a2dc8c'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:35.503870 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-9540fdbb-311e-4edd-a125-22c4fd124102 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b34040676bdb4d5f90488f31469fa0f1', 'request_id': 'req-9540fdbb-311e-4edd-a125-22c4fd124102'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:35.521968 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-9540fdbb-311e-4edd-a125-22c4fd124102 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b34040676bdb4d5f90488f31469fa0f1', 'request_id': 'req-9540fdbb-311e-4edd-a125-22c4fd124102'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:35.522927 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-d7002ae8-d374-433f-8e07-6c85700fcbb2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80c36f803818485496548836f3a2dc8c'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:35.576467 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-d7002ae8-d374-433f-8e07-6c85700fcbb2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80c36f803818485496548836f3a2dc8c'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:35.576467 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-d7002ae8-d374-433f-8e07-6c85700fcbb2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80c36f803818485496548836f3a2dc8c'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:35.625065 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-9540fdbb-311e-4edd-a125-22c4fd124102 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b34040676bdb4d5f90488f31469fa0f1', 'request_id': 'req-9540fdbb-311e-4edd-a125-22c4fd124102'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:35.655788 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-9540fdbb-311e-4edd-a125-22c4fd124102 admin admin] The token cache is being invalidate because project b34040676bdb4d5f90488f31469fa0f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81517) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:07:35.656102 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-9540fdbb-311e-4edd-a125-22c4fd124102 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:35.659719 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-9540fdbb-311e-4edd-a125-22c4fd124102 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:35.665839 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 696/1359] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:07:35 2022] DELETE /identity/v3/projects/b34040676bdb4d5f90488f31469fa0f1 => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:07:35.674101 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155a6b07-ea34-40f0-8fa9-4cd5bc030824 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:35.674101 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155a6b07-ea34-40f0-8fa9-4cd5bc030824 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:35.674101 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155a6b07-ea34-40f0-8fa9-4cd5bc030824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:35.692682 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-d7002ae8-d374-433f-8e07-6c85700fcbb2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80c36f803818485496548836f3a2dc8c'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:35.699610 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-155a6b07-ea34-40f0-8fa9-4cd5bc030824 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:35.702259 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-155a6b07-ea34-40f0-8fa9-4cd5bc030824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:35.703308 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 664/1360] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:07:35 2022] DELETE /identity/v3/users/80c36f803818485496548836f3a2dc8c => generated 0 bytes in 606 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 03:07:35.825829 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 697/1361] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:07:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:07:35.830179 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd488610-4d9f-4377-a277-dbb6056accc7 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:35.830179 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd488610-4d9f-4377-a277-dbb6056accc7 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:35.830526 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd488610-4d9f-4377-a277-dbb6056accc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:35.867353 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-bd488610-4d9f-4377-a277-dbb6056accc7 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:35.868900 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-bd488610-4d9f-4377-a277-dbb6056accc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:35.894868 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 665/1362] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:07:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:07:35.899073 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155a6b07-ea34-40f0-8fa9-4cd5bc030824 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:35.915191 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155a6b07-ea34-40f0-8fa9-4cd5bc030824 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:35.924014 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b3918c-3d3f-4f7d-b560-88804dd4208a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:35.925057 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b3918c-3d3f-4f7d-b560-88804dd4208a admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:35.925556 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b3918c-3d3f-4f7d-b560-88804dd4208a admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:35.926290 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b3918c-3d3f-4f7d-b560-88804dd4208a admin admin] PATH_INFO: `/v3/domains` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:35.961891 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b3918c-3d3f-4f7d-b560-88804dd4208a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 03:07:35.970883 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b3918c-3d3f-4f7d-b560-88804dd4208a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:35.970883 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b3918c-3d3f-4f7d-b560-88804dd4208a admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:35.993831 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 698/1363] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Oct 14 03:07:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 03:07:35.999282 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd488610-4d9f-4377-a277-dbb6056accc7 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:36.018937 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd488610-4d9f-4377-a277-dbb6056accc7 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:36.062432 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e998a4d3-604c-40c2-a03c-aadc11f21c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:36.070754 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e998a4d3-604c-40c2-a03c-aadc11f21c9b admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:36.071526 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e998a4d3-604c-40c2-a03c-aadc11f21c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:36.073935 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e998a4d3-604c-40c2-a03c-aadc11f21c9b admin admin] PATH_INFO: `/v3/projects` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:36.074997 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e998a4d3-604c-40c2-a03c-aadc11f21c9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:36.077192 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e998a4d3-604c-40c2-a03c-aadc11f21c9b admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:36.095649 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b3918c-3d3f-4f7d-b560-88804dd4208a admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:36.097983 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-87b3918c-3d3f-4f7d-b560-88804dd4208a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:36.115052 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b3918c-3d3f-4f7d-b560-88804dd4208a admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:36.121951 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064a55c9-f7db-4af6-988f-2de810a1af78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:36.132867 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064a55c9-f7db-4af6-988f-2de810a1af78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:36.133164 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064a55c9-f7db-4af6-988f-2de810a1af78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:36.133546 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064a55c9-f7db-4af6-988f-2de810a1af78 admin admin] PATH_INFO: `/v3/projects/c7f6c46584ea4ff19b771019b48c8811` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:36.135155 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064a55c9-f7db-4af6-988f-2de810a1af78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7f6c46584ea4ff19b771019b48c8811)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:36.137329 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064a55c9-f7db-4af6-988f-2de810a1af78 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:36.228993 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 666/1364] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:07:35 2022] POST /identity/v3/projects => generated 367 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:07:36.233823 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e998a4d3-604c-40c2-a03c-aadc11f21c9b admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:36.284858 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-064a55c9-f7db-4af6-988f-2de810a1af78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7f6c46584ea4ff19b771019b48c8811', 'request_id': 'req-064a55c9-f7db-4af6-988f-2de810a1af78'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:36.299926 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-064a55c9-f7db-4af6-988f-2de810a1af78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7f6c46584ea4ff19b771019b48c8811', 'request_id': 'req-064a55c9-f7db-4af6-988f-2de810a1af78'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:36.351850 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e998a4d3-604c-40c2-a03c-aadc11f21c9b admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:36.424256 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-064a55c9-f7db-4af6-988f-2de810a1af78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7f6c46584ea4ff19b771019b48c8811', 'request_id': 'req-064a55c9-f7db-4af6-988f-2de810a1af78'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:36.448391 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-064a55c9-f7db-4af6-988f-2de810a1af78 admin admin] The token cache is being invalidate because project c7f6c46584ea4ff19b771019b48c8811 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81517) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:07:36.449069 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-064a55c9-f7db-4af6-988f-2de810a1af78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:36.452296 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-064a55c9-f7db-4af6-988f-2de810a1af78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:36.455657 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 699/1365] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:07:36 2022] DELETE /identity/v3/projects/c7f6c46584ea4ff19b771019b48c8811 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:07:36.460310 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb96c90-e7f9-48ce-9307-6f5160643c44 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:36.460890 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb96c90-e7f9-48ce-9307-6f5160643c44 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:36.461400 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb96c90-e7f9-48ce-9307-6f5160643c44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:36.491324 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f5f411-cde7-4dc7-b04b-ebc840b3067b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:36.492272 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f5f411-cde7-4dc7-b04b-ebc840b3067b admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:36.492718 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f5f411-cde7-4dc7-b04b-ebc840b3067b admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:36.493151 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f5f411-cde7-4dc7-b04b-ebc840b3067b admin admin] PATH_INFO: `/v3/users` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:36.496003 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f5f411-cde7-4dc7-b04b-ebc840b3067b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:36.498669 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f5f411-cde7-4dc7-b04b-ebc840b3067b admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:36.506243 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-bbb96c90-e7f9-48ce-9307-6f5160643c44 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:36.509515 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-bbb96c90-e7f9-48ce-9307-6f5160643c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:36.533819 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 700/1366] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:07:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 03:07:36.538985 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c857195-c048-4298-8858-1010ed26570a None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:36.541269 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c857195-c048-4298-8858-1010ed26570a None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:36.541779 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c857195-c048-4298-8858-1010ed26570a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:36.580663 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-5c857195-c048-4298-8858-1010ed26570a None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:36.582898 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-5c857195-c048-4298-8858-1010ed26570a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:36.609485 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 667/1367] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:07:36 2022] POST /identity/v3/users => generated 284 bytes in 376 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:07:36.613483 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f5f411-cde7-4dc7-b04b-ebc840b3067b admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:36.617530 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-71f5f411-cde7-4dc7-b04b-ebc840b3067b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:36.634097 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 701/1368] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:07:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:07:36.636906 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c857195-c048-4298-8858-1010ed26570a None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:36.657503 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f5f411-cde7-4dc7-b04b-ebc840b3067b admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:36.662289 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c857195-c048-4298-8858-1010ed26570a None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:36.669564 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fc0446-317e-4b4c-a540-d52e90b98279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:36.670523 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fc0446-317e-4b4c-a540-d52e90b98279 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:36.671090 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fc0446-317e-4b4c-a540-d52e90b98279 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:36.671565 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fc0446-317e-4b4c-a540-d52e90b98279 admin admin] PATH_INFO: `/v3/domains` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:36.672532 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fc0446-317e-4b4c-a540-d52e90b98279 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 03:07:36.673118 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fc0446-317e-4b4c-a540-d52e90b98279 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:36.676891 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fc0446-317e-4b4c-a540-d52e90b98279 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:36.679726 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e81a78-b338-4460-aa9e-9473e4d73393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:36.680573 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e81a78-b338-4460-aa9e-9473e4d73393 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:36.684404 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e81a78-b338-4460-aa9e-9473e4d73393 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:36.684404 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e81a78-b338-4460-aa9e-9473e4d73393 admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:36.684404 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e81a78-b338-4460-aa9e-9473e4d73393 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:36.685334 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e81a78-b338-4460-aa9e-9473e4d73393 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:36.707486 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 702/1369] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Oct 14 03:07:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 03:07:36.708749 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fc0446-317e-4b4c-a540-d52e90b98279 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:36.714369 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 668/1370] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:07:36 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:07:36.717942 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e81a78-b338-4460-aa9e-9473e4d73393 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:36.722671 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fc0446-317e-4b4c-a540-d52e90b98279 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:36.729015 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3722fd89-fdb3-4fd6-8c58-bc6a0d3880e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:36.729810 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3722fd89-fdb3-4fd6-8c58-bc6a0d3880e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:36.730346 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3722fd89-fdb3-4fd6-8c58-bc6a0d3880e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:36.731461 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3722fd89-fdb3-4fd6-8c58-bc6a0d3880e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:36.732256 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3722fd89-fdb3-4fd6-8c58-bc6a0d3880e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:36.734961 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3722fd89-fdb3-4fd6-8c58-bc6a0d3880e3 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:36.737606 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e81a78-b338-4460-aa9e-9473e4d73393 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:36.752440 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eadc76-6a48-4731-9b64-c7770cce65ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:36.753356 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eadc76-6a48-4731-9b64-c7770cce65ca admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:36.753915 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eadc76-6a48-4731-9b64-c7770cce65ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:36.754388 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eadc76-6a48-4731-9b64-c7770cce65ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:36.755777 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4eadc76-6a48-4731-9b64-c7770cce65ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:36.757642 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4eadc76-6a48-4731-9b64-c7770cce65ca admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:36.786636 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 669/1371] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:07:36 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 03:07:36.790681 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eadc76-6a48-4731-9b64-c7770cce65ca admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:36.809195 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eadc76-6a48-4731-9b64-c7770cce65ca admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:36.822885 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2a221f-a4c8-4aaf-a1b1-f29b385d6663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:36.823652 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2a221f-a4c8-4aaf-a1b1-f29b385d6663 admin admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:36.823944 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2a221f-a4c8-4aaf-a1b1-f29b385d6663 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:36.824325 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2a221f-a4c8-4aaf-a1b1-f29b385d6663 admin admin] PATH_INFO: `/v3/projects/f7140b54e63c4d5e8df904f73c0192c2/users/07afc8012c964bf39039dcf26ee38186/roles/9bd0470b544240209a420f640c79c06c` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:36.826734 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:07:36.837273 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 703/1372] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:07:36 2022] POST /identity/v3/projects => generated 363 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:07:36.839066 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3722fd89-fdb3-4fd6-8c58-bc6a0d3880e3 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:36.865339 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2a221f-a4c8-4aaf-a1b1-f29b385d6663 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7140b54e63c4d5e8df904f73c0192c2, user_id=07afc8012c964bf39039dcf26ee38186, role_id=9bd0470b544240209a420f640c79c06c)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:36.871797 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2a221f-a4c8-4aaf-a1b1-f29b385d6663 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:36.934171 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 670/1373] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:07:36 2022] PUT /identity/v3/projects/f7140b54e63c4d5e8df904f73c0192c2/users/07afc8012c964bf39039dcf26ee38186/roles/9bd0470b544240209a420f640c79c06c => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 03:07:36.937642 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2a221f-a4c8-4aaf-a1b1-f29b385d6663 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:36.938043 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-9e2a221f-a4c8-4aaf-a1b1-f29b385d6663 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:36.948422 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3722fd89-fdb3-4fd6-8c58-bc6a0d3880e3 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:36.952393 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:07:37.057612 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6381b56e-a0ce-48b1-a24b-5e855afde47f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:37.058317 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6381b56e-a0ce-48b1-a24b-5e855afde47f admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:37.058606 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6381b56e-a0ce-48b1-a24b-5e855afde47f admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:37.059582 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6381b56e-a0ce-48b1-a24b-5e855afde47f admin admin] PATH_INFO: `/v3/users` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:37.063649 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6381b56e-a0ce-48b1-a24b-5e855afde47f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:37.070636 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6381b56e-a0ce-48b1-a24b-5e855afde47f admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:37.077964 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2a221f-a4c8-4aaf-a1b1-f29b385d6663 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:37.094396 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aa5b18-dc54-41e4-8b77-8fed8f26dfd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:37.099338 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aa5b18-dc54-41e4-8b77-8fed8f26dfd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:37.099338 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aa5b18-dc54-41e4-8b77-8fed8f26dfd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:37.099338 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aa5b18-dc54-41e4-8b77-8fed8f26dfd2 admin admin] PATH_INFO: `/v3/projects/45877a0d46fb4028aedbe0eb7ac28dc9` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:37.101116 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38aa5b18-dc54-41e4-8b77-8fed8f26dfd2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45877a0d46fb4028aedbe0eb7ac28dc9)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:37.104268 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38aa5b18-dc54-41e4-8b77-8fed8f26dfd2 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:37.179068 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 704/1374] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:07:36 2022] POST /identity/v3/users => generated 282 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:07:37.180672 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559f080b-d09a-4164-9b65-3b71e27da394 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:37.180974 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559f080b-d09a-4164-9b65-3b71e27da394 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:37.181263 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559f080b-d09a-4164-9b65-3b71e27da394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:37.236899 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-559f080b-d09a-4164-9b65-3b71e27da394 None None] MFA Rules not processed for user `07afc8012c964bf39039dcf26ee38186`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:37.242634 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-559f080b-d09a-4164-9b65-3b71e27da394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:37.259153 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-38aa5b18-dc54-41e4-8b77-8fed8f26dfd2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45877a0d46fb4028aedbe0eb7ac28dc9', 'request_id': 'req-38aa5b18-dc54-41e4-8b77-8fed8f26dfd2'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:37.275012 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-38aa5b18-dc54-41e4-8b77-8fed8f26dfd2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45877a0d46fb4028aedbe0eb7ac28dc9', 'request_id': 'req-38aa5b18-dc54-41e4-8b77-8fed8f26dfd2'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:37.347684 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 705/1375] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:07:37 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:07:37.348956 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559f080b-d09a-4164-9b65-3b71e27da394 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:37.354288 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-38aa5b18-dc54-41e4-8b77-8fed8f26dfd2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45877a0d46fb4028aedbe0eb7ac28dc9', 'request_id': 'req-38aa5b18-dc54-41e4-8b77-8fed8f26dfd2'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:37.375079 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-38aa5b18-dc54-41e4-8b77-8fed8f26dfd2 admin admin] The token cache is being invalidate because project 45877a0d46fb4028aedbe0eb7ac28dc9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:07:37.375365 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-38aa5b18-dc54-41e4-8b77-8fed8f26dfd2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:37.377272 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-38aa5b18-dc54-41e4-8b77-8fed8f26dfd2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:37.381009 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 671/1376] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:07:36 2022] DELETE /identity/v3/projects/45877a0d46fb4028aedbe0eb7ac28dc9 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:07:37.381009 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aa5b18-dc54-41e4-8b77-8fed8f26dfd2 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:37.388333 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-38aa5b18-dc54-41e4-8b77-8fed8f26dfd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:37.476147 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559f080b-d09a-4164-9b65-3b71e27da394 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:37.484746 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8003b9bd-d894-496f-bda2-8fe0a3b288d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:37.485378 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8003b9bd-d894-496f-bda2-8fe0a3b288d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:37.485660 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8003b9bd-d894-496f-bda2-8fe0a3b288d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:37.485960 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8003b9bd-d894-496f-bda2-8fe0a3b288d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:37.486928 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8003b9bd-d894-496f-bda2-8fe0a3b288d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:37.488285 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8003b9bd-d894-496f-bda2-8fe0a3b288d3 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:37.511293 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 706/1377] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:07:37 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:07:37.512701 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c24ba-a882-4ca8-9933-15c6e0f98eda None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:37.512997 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c24ba-a882-4ca8-9933-15c6e0f98eda None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:37.513277 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c24ba-a882-4ca8-9933-15c6e0f98eda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:37.531577 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aa5b18-dc54-41e4-8b77-8fed8f26dfd2 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:37.544109 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9657874c-468d-4970-8f8d-74de6be69c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:37.544422 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9657874c-468d-4970-8f8d-74de6be69c37 admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:37.544667 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9657874c-468d-4970-8f8d-74de6be69c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:37.545023 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9657874c-468d-4970-8f8d-74de6be69c37 admin admin] PATH_INFO: `/v3/projects` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:37.546057 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9657874c-468d-4970-8f8d-74de6be69c37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:37.554763 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9657874c-468d-4970-8f8d-74de6be69c37 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:37.577368 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-cf8c24ba-a882-4ca8-9933-15c6e0f98eda None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:37.579031 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-cf8c24ba-a882-4ca8-9933-15c6e0f98eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:37.604369 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 707/1378] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:07:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:07:37.605094 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8c24ba-a882-4ca8-9933-15c6e0f98eda None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:37.607143 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-cf8c24ba-a882-4ca8-9933-15c6e0f98eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:37.637964 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8c24ba-a882-4ca8-9933-15c6e0f98eda None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:37.649795 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653c2efa-e01e-44d2-a6f7-1b6e1adceb07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:37.650684 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653c2efa-e01e-44d2-a6f7-1b6e1adceb07 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:37.650684 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653c2efa-e01e-44d2-a6f7-1b6e1adceb07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:37.650882 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653c2efa-e01e-44d2-a6f7-1b6e1adceb07 admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:37.651441 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653c2efa-e01e-44d2-a6f7-1b6e1adceb07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:37.653380 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653c2efa-e01e-44d2-a6f7-1b6e1adceb07 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:37.672343 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 672/1379] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:07:37 2022] POST /identity/v3/projects => generated 369 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:07:37.676940 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 708/1380] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:07:37 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:07:37.678410 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c36993-ae20-4dd3-a2d8-eeb51f1babcd None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:37.679079 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c36993-ae20-4dd3-a2d8-eeb51f1babcd None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:37.679285 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653c2efa-e01e-44d2-a6f7-1b6e1adceb07 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:37.687032 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c36993-ae20-4dd3-a2d8-eeb51f1babcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:37.703614 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:07:37.705985 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653c2efa-e01e-44d2-a6f7-1b6e1adceb07 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:37.749929 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-64c36993-ae20-4dd3-a2d8-eeb51f1babcd None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:37.751909 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-64c36993-ae20-4dd3-a2d8-eeb51f1babcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:37.828028 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c663cf51-7ef5-4d8c-b4db-d6f329573eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:37.828651 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c663cf51-7ef5-4d8c-b4db-d6f329573eaa admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:37.828933 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c663cf51-7ef5-4d8c-b4db-d6f329573eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:37.829209 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c663cf51-7ef5-4d8c-b4db-d6f329573eaa admin admin] PATH_INFO: `/v3/users` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:37.829905 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c663cf51-7ef5-4d8c-b4db-d6f329573eaa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:37.831842 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c663cf51-7ef5-4d8c-b4db-d6f329573eaa admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:37.870122 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:07:37.917914 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 673/1381] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:07:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:07:37.919409 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c36993-ae20-4dd3-a2d8-eeb51f1babcd None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:37.967093 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 709/1382] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:07:37 2022] POST /identity/v3/users => generated 291 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:07:37.968158 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c663cf51-7ef5-4d8c-b4db-d6f329573eaa admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:37.968276 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c36993-ae20-4dd3-a2d8-eeb51f1babcd None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:37.994809 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e56c1b-28b0-4d0e-9fbe-37ef28be9a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:37.995626 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e56c1b-28b0-4d0e-9fbe-37ef28be9a12 admin admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:37.996361 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e56c1b-28b0-4d0e-9fbe-37ef28be9a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:37.998590 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e56c1b-28b0-4d0e-9fbe-37ef28be9a12 admin admin] PATH_INFO: `/v3/projects/872d030527f5428489100defa400a9ba/users/c357d22167894543ad8e9648ea7894d7/roles/9bd0470b544240209a420f640c79c06c` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:37.999474 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c663cf51-7ef5-4d8c-b4db-d6f329573eaa admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:38.032353 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1a746d-e601-4e7b-aa69-f1e6e1dd8ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:38.033091 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1a746d-e601-4e7b-aa69-f1e6e1dd8ce0 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:38.033379 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1a746d-e601-4e7b-aa69-f1e6e1dd8ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:38.033651 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1a746d-e601-4e7b-aa69-f1e6e1dd8ce0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:38.034386 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1a746d-e601-4e7b-aa69-f1e6e1dd8ce0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 03:07:38.039546 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1a746d-e601-4e7b-aa69-f1e6e1dd8ce0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:38.039546 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1a746d-e601-4e7b-aa69-f1e6e1dd8ce0 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:38.040478 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e56c1b-28b0-4d0e-9fbe-37ef28be9a12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=872d030527f5428489100defa400a9ba, user_id=c357d22167894543ad8e9648ea7894d7, role_id=9bd0470b544240209a420f640c79c06c)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:38.044620 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e56c1b-28b0-4d0e-9fbe-37ef28be9a12 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:38.077039 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 710/1383] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Oct 14 03:07:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 03:07:38.077039 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1a746d-e601-4e7b-aa69-f1e6e1dd8ce0 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:38.107165 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1a746d-e601-4e7b-aa69-f1e6e1dd8ce0 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:38.119340 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3597de28-f573-434a-973d-ba89951fa5db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:38.119659 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 674/1384] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:07:37 2022] PUT /identity/v3/projects/872d030527f5428489100defa400a9ba/users/c357d22167894543ad8e9648ea7894d7/roles/9bd0470b544240209a420f640c79c06c => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:07:38.119942 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3597de28-f573-434a-973d-ba89951fa5db admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:38.120227 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3597de28-f573-434a-973d-ba89951fa5db admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:38.120504 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3597de28-f573-434a-973d-ba89951fa5db admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:38.120620 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e56c1b-28b0-4d0e-9fbe-37ef28be9a12 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:38.121158 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3597de28-f573-434a-973d-ba89951fa5db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:38.123026 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3597de28-f573-434a-973d-ba89951fa5db admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:38.154530 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 711/1385] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:07:38 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:07:38.156029 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeb787f-1399-493b-96f5-c94dade6bb76 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:38.160134 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeb787f-1399-493b-96f5-c94dade6bb76 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:38.160134 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeb787f-1399-493b-96f5-c94dade6bb76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:38.230311 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-4eeb787f-1399-493b-96f5-c94dade6bb76 None None] MFA Rules not processed for user `c357d22167894543ad8e9648ea7894d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:38.231558 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-4eeb787f-1399-493b-96f5-c94dade6bb76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:38.283101 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e56c1b-28b0-4d0e-9fbe-37ef28be9a12 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:38.297011 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0d9bbe-85cd-474e-915d-86ba684df17f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:38.308768 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0d9bbe-85cd-474e-915d-86ba684df17f admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:38.308768 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0d9bbe-85cd-474e-915d-86ba684df17f admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:38.308768 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0d9bbe-85cd-474e-915d-86ba684df17f admin admin] PATH_INFO: `/v3/projects` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:38.308768 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0d9bbe-85cd-474e-915d-86ba684df17f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:38.308768 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0d9bbe-85cd-474e-915d-86ba684df17f admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:38.337267 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:07:38.384564 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:07:38.407801 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 712/1386] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:07:38 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:07:38.407801 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeb787f-1399-493b-96f5-c94dade6bb76 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:38.412988 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 675/1387] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:07:38 2022] POST /identity/v3/projects => generated 371 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:07:38.416622 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0d9bbe-85cd-474e-915d-86ba684df17f admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:38.445464 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeb787f-1399-493b-96f5-c94dade6bb76 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:38.596717 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0d9bbe-85cd-474e-915d-86ba684df17f admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:38.613191 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80a3eac-cef4-4a1e-9f01-c306ba6ed41f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:38.613846 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80a3eac-cef4-4a1e-9f01-c306ba6ed41f admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:38.614131 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80a3eac-cef4-4a1e-9f01-c306ba6ed41f admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:38.614405 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80a3eac-cef4-4a1e-9f01-c306ba6ed41f admin admin] PATH_INFO: `/v3/projects` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:38.617809 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80a3eac-cef4-4a1e-9f01-c306ba6ed41f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:38.623142 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80a3eac-cef4-4a1e-9f01-c306ba6ed41f admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:38.655720 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0510091-d0e2-4697-be45-53ee8e7d634c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:38.658572 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0510091-d0e2-4697-be45-53ee8e7d634c admin admin] REQUEST_METHOD: `PUT` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:38.658861 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0510091-d0e2-4697-be45-53ee8e7d634c admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:38.659151 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0510091-d0e2-4697-be45-53ee8e7d634c admin admin] PATH_INFO: `/v3/projects/d7d00636950b44e7b6a4ef7496bae560/users/4c3c26f22bb548ce99075c49c79443a4/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:38.722999 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0510091-d0e2-4697-be45-53ee8e7d634c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7d00636950b44e7b6a4ef7496bae560, user_id=4c3c26f22bb548ce99075c49c79443a4, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:38.731066 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0510091-d0e2-4697-be45-53ee8e7d634c admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:38.796408 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 676/1388] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:07:38 2022] POST /identity/v3/projects => generated 363 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:07:38.798189 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80a3eac-cef4-4a1e-9f01-c306ba6ed41f admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:38.847807 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 713/1389] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:07:38 2022] PUT /identity/v3/projects/d7d00636950b44e7b6a4ef7496bae560/users/4c3c26f22bb548ce99075c49c79443a4/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:07:38.851150 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0510091-d0e2-4697-be45-53ee8e7d634c admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:39.131114 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80a3eac-cef4-4a1e-9f01-c306ba6ed41f admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:39.139246 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0510091-d0e2-4697-be45-53ee8e7d634c admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:39.139246 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f370f6-2e27-43f5-9e2e-5e68781ef414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:39.139246 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f370f6-2e27-43f5-9e2e-5e68781ef414 admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:39.139246 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f370f6-2e27-43f5-9e2e-5e68781ef414 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:39.145118 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f370f6-2e27-43f5-9e2e-5e68781ef414 admin admin] PATH_INFO: `/v3/users` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:39.145118 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f370f6-2e27-43f5-9e2e-5e68781ef414 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:39.145118 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f370f6-2e27-43f5-9e2e-5e68781ef414 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:39.219709 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea055f12-9a5a-4693-82cb-6a374adc1d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:39.220355 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea055f12-9a5a-4693-82cb-6a374adc1d42 admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:39.220646 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea055f12-9a5a-4693-82cb-6a374adc1d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:39.220920 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea055f12-9a5a-4693-82cb-6a374adc1d42 admin admin] PATH_INFO: `/v3/users` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:39.221751 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea055f12-9a5a-4693-82cb-6a374adc1d42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:39.231228 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea055f12-9a5a-4693-82cb-6a374adc1d42 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:39.344130 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:07:39.350890 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 714/1390] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:07:38 2022] POST /identity/v3/users => generated 288 bytes in 502 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:07:39.352156 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f370f6-2e27-43f5-9e2e-5e68781ef414 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:39.378461 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f370f6-2e27-43f5-9e2e-5e68781ef414 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:39.378596 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 677/1391] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:07:38 2022] POST /identity/v3/users => generated 286 bytes in 582 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:07:39.381021 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea055f12-9a5a-4693-82cb-6a374adc1d42 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:39.392476 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7ebb2b-b2a7-4bf6-a02c-21d9512d1f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:39.393496 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ebb2b-b2a7-4bf6-a02c-21d9512d1f04 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:39.393628 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ebb2b-b2a7-4bf6-a02c-21d9512d1f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:39.393931 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ebb2b-b2a7-4bf6-a02c-21d9512d1f04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:39.395978 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7ebb2b-b2a7-4bf6-a02c-21d9512d1f04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:39.402003 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7ebb2b-b2a7-4bf6-a02c-21d9512d1f04 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:39.409884 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea055f12-9a5a-4693-82cb-6a374adc1d42 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:39.429725 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6faa59e-769b-4fdd-9814-0743e4b9580a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:39.432773 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6faa59e-769b-4fdd-9814-0743e4b9580a admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:39.432773 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6faa59e-769b-4fdd-9814-0743e4b9580a admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:39.432773 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6faa59e-769b-4fdd-9814-0743e4b9580a admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:39.433521 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6faa59e-769b-4fdd-9814-0743e4b9580a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:39.434842 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:07:39.437402 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6faa59e-769b-4fdd-9814-0743e4b9580a admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:39.460438 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 715/1392] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:07:39 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:07:39.460438 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7ebb2b-b2a7-4bf6-a02c-21d9512d1f04 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:39.472962 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 678/1393] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:07:39 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:07:39.477030 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6faa59e-769b-4fdd-9814-0743e4b9580a admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:39.487599 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7ebb2b-b2a7-4bf6-a02c-21d9512d1f04 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:39.503607 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6faa59e-769b-4fdd-9814-0743e4b9580a admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:39.506879 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93299516-8200-4ed2-a8cd-0cb226cbf209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:39.507685 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93299516-8200-4ed2-a8cd-0cb226cbf209 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:39.508130 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93299516-8200-4ed2-a8cd-0cb226cbf209 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:39.508588 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93299516-8200-4ed2-a8cd-0cb226cbf209 admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:39.509393 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93299516-8200-4ed2-a8cd-0cb226cbf209 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:39.511355 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93299516-8200-4ed2-a8cd-0cb226cbf209 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:39.515212 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f117ca-8eb1-4604-83ba-461efea2f403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:39.515931 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f117ca-8eb1-4604-83ba-461efea2f403 admin admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:39.516216 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f117ca-8eb1-4604-83ba-461efea2f403 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:39.516494 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f117ca-8eb1-4604-83ba-461efea2f403 admin admin] PATH_INFO: `/v3/projects/d7d00636950b44e7b6a4ef7496bae560/users/4c3c26f22bb548ce99075c49c79443a4/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:39.520120 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f117ca-8eb1-4604-83ba-461efea2f403 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7d00636950b44e7b6a4ef7496bae560, user_id=4c3c26f22bb548ce99075c49c79443a4, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:39.533467 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f117ca-8eb1-4604-83ba-461efea2f403 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:39.540499 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 716/1394] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:07:39 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:07:39.542701 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93299516-8200-4ed2-a8cd-0cb226cbf209 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:39.559115 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93299516-8200-4ed2-a8cd-0cb226cbf209 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:39.580215 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 679/1395] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:07:39 2022] PUT /identity/v3/projects/d7d00636950b44e7b6a4ef7496bae560/users/4c3c26f22bb548ce99075c49c79443a4/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:07:39.580215 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f117ca-8eb1-4604-83ba-461efea2f403 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:39.726158 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f117ca-8eb1-4604-83ba-461efea2f403 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:39.731850 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b08641-6868-4e99-afad-9c6a2829a72d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:39.732597 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b08641-6868-4e99-afad-9c6a2829a72d admin admin] REQUEST_METHOD: `PUT` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:39.732886 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b08641-6868-4e99-afad-9c6a2829a72d admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:39.733395 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b08641-6868-4e99-afad-9c6a2829a72d admin admin] PATH_INFO: `/v3/projects/e9eae27dea844c47841f2a16a4325ab8/users/6b05fa1cc14b4e7db2f19d0a24b98be0/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:39.738291 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b14c69-a460-4977-a24b-72cf9b0f6635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:39.738966 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b14c69-a460-4977-a24b-72cf9b0f6635 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:39.739251 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b14c69-a460-4977-a24b-72cf9b0f6635 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:39.739575 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b14c69-a460-4977-a24b-72cf9b0f6635 admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:39.741151 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b14c69-a460-4977-a24b-72cf9b0f6635 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:39.742917 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b14c69-a460-4977-a24b-72cf9b0f6635 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:39.762148 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 680/1396] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:07:39 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:07:39.768173 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce548cd0-cfd9-483f-8fef-f6cb558a1b06 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:39.768982 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce548cd0-cfd9-483f-8fef-f6cb558a1b06 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:39.769271 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce548cd0-cfd9-483f-8fef-f6cb558a1b06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:39.788114 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b08641-6868-4e99-afad-9c6a2829a72d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9eae27dea844c47841f2a16a4325ab8, user_id=6b05fa1cc14b4e7db2f19d0a24b98be0, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:39.793110 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b08641-6868-4e99-afad-9c6a2829a72d admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:39.835951 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-ce548cd0-cfd9-483f-8fef-f6cb558a1b06 None None] MFA Rules not processed for user `4c3c26f22bb548ce99075c49c79443a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:39.837076 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-ce548cd0-cfd9-483f-8fef-f6cb558a1b06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:39.879322 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 717/1397] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:07:39 2022] PUT /identity/v3/projects/e9eae27dea844c47841f2a16a4325ab8/users/6b05fa1cc14b4e7db2f19d0a24b98be0/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:07:39.880442 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b08641-6868-4e99-afad-9c6a2829a72d admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:40.018099 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:07:40.051192 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b08641-6868-4e99-afad-9c6a2829a72d admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:40.061453 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 681/1398] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:07:39 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:07:40.063255 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce548cd0-cfd9-483f-8fef-f6cb558a1b06 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:40.081322 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4bb3e7-3fe2-4700-bb9e-82692b7306b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:40.081668 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4bb3e7-3fe2-4700-bb9e-82692b7306b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:40.081987 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4bb3e7-3fe2-4700-bb9e-82692b7306b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:40.082267 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4bb3e7-3fe2-4700-bb9e-82692b7306b9 admin admin] PATH_INFO: `/v3/projects/81ac2ef8804b47d785f51e605731ad5b/users/c3cd74e2ed67443abef1d66a2779a083/roles/9bd0470b544240209a420f640c79c06c` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:40.089187 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce548cd0-cfd9-483f-8fef-f6cb558a1b06 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:40.106255 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cb42c5-3168-4e7d-8126-d95a82b1f53a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:40.106887 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cb42c5-3168-4e7d-8126-d95a82b1f53a admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:40.108287 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cb42c5-3168-4e7d-8126-d95a82b1f53a admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:40.108287 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cb42c5-3168-4e7d-8126-d95a82b1f53a admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:40.108675 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cb42c5-3168-4e7d-8126-d95a82b1f53a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:40.113325 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cb42c5-3168-4e7d-8126-d95a82b1f53a admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:40.132652 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4bb3e7-3fe2-4700-bb9e-82692b7306b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81ac2ef8804b47d785f51e605731ad5b, user_id=c3cd74e2ed67443abef1d66a2779a083, role_id=9bd0470b544240209a420f640c79c06c)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:40.137987 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4bb3e7-3fe2-4700-bb9e-82692b7306b9 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:40.147687 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 682/1399] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:07:40 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:07:40.148743 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cb42c5-3168-4e7d-8126-d95a82b1f53a admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:40.151867 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-23cb42c5-3168-4e7d-8126-d95a82b1f53a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:40.207102 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 718/1400] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:07:39 2022] PUT /identity/v3/projects/81ac2ef8804b47d785f51e605731ad5b/users/c3cd74e2ed67443abef1d66a2779a083/roles/9bd0470b544240209a420f640c79c06c => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:07:40.218509 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:07:40.221685 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4bb3e7-3fe2-4700-bb9e-82692b7306b9 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:40.306945 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cb42c5-3168-4e7d-8126-d95a82b1f53a admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:40.345198 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:07:40.374024 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4bb3e7-3fe2-4700-bb9e-82692b7306b9 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:40.384158 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3992e5-5d45-45a6-ab6c-26116e7f6b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:40.385790 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3992e5-5d45-45a6-ab6c-26116e7f6b3d admin admin] REQUEST_METHOD: `PUT` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:40.386272 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3992e5-5d45-45a6-ab6c-26116e7f6b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:40.386610 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3992e5-5d45-45a6-ab6c-26116e7f6b3d admin admin] PATH_INFO: `/v3/projects/e9eae27dea844c47841f2a16a4325ab8/users/6b05fa1cc14b4e7db2f19d0a24b98be0/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:40.388832 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3992e5-5d45-45a6-ab6c-26116e7f6b3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9eae27dea844c47841f2a16a4325ab8, user_id=6b05fa1cc14b4e7db2f19d0a24b98be0, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:40.391971 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3992e5-5d45-45a6-ab6c-26116e7f6b3d admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:40.413949 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 719/1401] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:07:40 2022] PUT /identity/v3/projects/e9eae27dea844c47841f2a16a4325ab8/users/6b05fa1cc14b4e7db2f19d0a24b98be0/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:07:40.413949 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8444f06e-7e39-4aca-a8f9-608b4d794ab6 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:40.413949 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8444f06e-7e39-4aca-a8f9-608b4d794ab6 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:40.413949 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8444f06e-7e39-4aca-a8f9-608b4d794ab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:40.445191 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9e761a-6293-474b-88ee-13c9901e36a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab599d98b0624a2e89090312bcab713f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:40.445908 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9e761a-6293-474b-88ee-13c9901e36a8 service cinder] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:40.446262 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9e761a-6293-474b-88ee-13c9901e36a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:40.446476 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9e761a-6293-474b-88ee-13c9901e36a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:40.454890 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-8444f06e-7e39-4aca-a8f9-608b4d794ab6 None None] MFA Rules not processed for user `c3cd74e2ed67443abef1d66a2779a083`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:40.456736 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-8444f06e-7e39-4aca-a8f9-608b4d794ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:40.582573 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9e761a-6293-474b-88ee-13c9901e36a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:40.583943 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9e761a-6293-474b-88ee-13c9901e36a8 service cinder] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:40.606997 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 683/1402] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:07:40 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:07:40.609433 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e610371e-e45c-47b5-906e-5ce26dbfaef7 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:40.609723 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e610371e-e45c-47b5-906e-5ce26dbfaef7 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:40.612601 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e610371e-e45c-47b5-906e-5ce26dbfaef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:40.689045 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-e610371e-e45c-47b5-906e-5ce26dbfaef7 None None] MFA Rules not processed for user `6b05fa1cc14b4e7db2f19d0a24b98be0`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:40.692221 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-e610371e-e45c-47b5-906e-5ce26dbfaef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:40.710898 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 720/1403] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:07:40 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:07:40.713526 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfac960-1e26-47c9-b306-248e1a7f49a4 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:40.714042 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfac960-1e26-47c9-b306-248e1a7f49a4 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:40.715128 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfac960-1e26-47c9-b306-248e1a7f49a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:40.808834 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-7bfac960-1e26-47c9-b306-248e1a7f49a4 None None] MFA Rules not processed for user `410b7c9e93c94ce78d1b5c0e646307a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:40.815691 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-7bfac960-1e26-47c9-b306-248e1a7f49a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:40.905549 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 684/1404] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:07:40 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:07:40.906681 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e610371e-e45c-47b5-906e-5ce26dbfaef7 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:40.946019 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:07:41.043013 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 721/1405] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:07:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:07:41.044108 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfac960-1e26-47c9-b306-248e1a7f49a4 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:41.097087 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e610371e-e45c-47b5-906e-5ce26dbfaef7 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:41.138288 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ca91a6-adbf-40cf-aaf6-c5b564671612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:41.141561 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ca91a6-adbf-40cf-aaf6-c5b564671612 admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:41.144276 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ca91a6-adbf-40cf-aaf6-c5b564671612 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:41.144841 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ca91a6-adbf-40cf-aaf6-c5b564671612 admin admin] PATH_INFO: `/v3/projects` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:41.147622 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ca91a6-adbf-40cf-aaf6-c5b564671612 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:41.156347 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ca91a6-adbf-40cf-aaf6-c5b564671612 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:41.253252 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfac960-1e26-47c9-b306-248e1a7f49a4 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:41.264946 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca376f33-31aa-4eb8-833a-5c5fb7f27a73 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab599d98b0624a2e89090312bcab713f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:41.265605 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca376f33-31aa-4eb8-833a-5c5fb7f27a73 service cinder] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:41.265919 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca376f33-31aa-4eb8-833a-5c5fb7f27a73 service cinder] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:41.266198 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca376f33-31aa-4eb8-833a-5c5fb7f27a73 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:41.292359 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 685/1406] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:07:40 2022] POST /identity/v3/projects => generated 373 bytes in 386 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:07:41.293283 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ca91a6-adbf-40cf-aaf6-c5b564671612 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:41.296907 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-70ca91a6-adbf-40cf-aaf6-c5b564671612 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:41.303085 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca376f33-31aa-4eb8-833a-5c5fb7f27a73 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:41.303085 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca376f33-31aa-4eb8-833a-5c5fb7f27a73 service cinder] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:41.317169 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:07:41.367836 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 722/1407] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:07:41 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:07:41.371964 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca376f33-31aa-4eb8-833a-5c5fb7f27a73 service cinder] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:41.437329 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca376f33-31aa-4eb8-833a-5c5fb7f27a73 service cinder] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:41.644815 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8297b89-fbc6-4602-b75c-f796bf3bdbbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:41.645471 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8297b89-fbc6-4602-b75c-f796bf3bdbbb admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:41.646807 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8297b89-fbc6-4602-b75c-f796bf3bdbbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:41.649438 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8297b89-fbc6-4602-b75c-f796bf3bdbbb admin admin] PATH_INFO: `/v3/users` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:41.650836 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8297b89-fbc6-4602-b75c-f796bf3bdbbb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:41.655568 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8297b89-fbc6-4602-b75c-f796bf3bdbbb admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:41.686440 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ca91a6-adbf-40cf-aaf6-c5b564671612 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:41.719961 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697717cd-2239-47e8-bad5-bf84a33bc392 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b07bf0faca47b28b50990d1720853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:41.722835 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697717cd-2239-47e8-bad5-bf84a33bc392 service nova] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:41.725692 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697717cd-2239-47e8-bad5-bf84a33bc392 service nova] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:41.728402 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697717cd-2239-47e8-bad5-bf84a33bc392 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:41.808987 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 723/1408] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:07:41 2022] POST /identity/v3/users => generated 293 bytes in 440 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:07:41.814170 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8297b89-fbc6-4602-b75c-f796bf3bdbbb admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:41.869296 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8297b89-fbc6-4602-b75c-f796bf3bdbbb admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:41.879004 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233012d9-55e0-418d-b1cf-290ac932133a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:41.879637 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233012d9-55e0-418d-b1cf-290ac932133a admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:41.880035 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233012d9-55e0-418d-b1cf-290ac932133a admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:41.880320 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233012d9-55e0-418d-b1cf-290ac932133a admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:41.881031 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233012d9-55e0-418d-b1cf-290ac932133a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:41.883348 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233012d9-55e0-418d-b1cf-290ac932133a admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:41.900638 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 724/1409] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:07:41 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:07:41.904016 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697717cd-2239-47e8-bad5-bf84a33bc392 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:41.904477 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233012d9-55e0-418d-b1cf-290ac932133a admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:41.906322 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697717cd-2239-47e8-bad5-bf84a33bc392 service nova] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:41.918297 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 686/1410] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:07:41 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 625 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:07:41.919221 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233012d9-55e0-418d-b1cf-290ac932133a admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:41.926237 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9ec8ca-3f9a-4b10-b148-70947632c1b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:41.927772 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9ec8ca-3f9a-4b10-b148-70947632c1b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:41.928336 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9ec8ca-3f9a-4b10-b148-70947632c1b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:41.928861 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9ec8ca-3f9a-4b10-b148-70947632c1b0 admin admin] PATH_INFO: `/v3/projects/7318cb48c37b44a9bcd1f857f936ecc1/users/9ed6a5af303b4142b82bc6333dcceac9/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:41.990479 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9ec8ca-3f9a-4b10-b148-70947632c1b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7318cb48c37b44a9bcd1f857f936ecc1, user_id=9ed6a5af303b4142b82bc6333dcceac9, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:41.999012 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9ec8ca-3f9a-4b10-b148-70947632c1b0 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:42.059962 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 725/1411] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:07:41 2022] PUT /identity/v3/projects/7318cb48c37b44a9bcd1f857f936ecc1/users/9ed6a5af303b4142b82bc6333dcceac9/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:07:42.064007 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697717cd-2239-47e8-bad5-bf84a33bc392 service nova] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:42.083434 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697717cd-2239-47e8-bad5-bf84a33bc392 service nova] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:42.104407 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7970e2-fc90-49be-a415-167a86fe468b None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:42.106265 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7970e2-fc90-49be-a415-167a86fe468b None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:42.106265 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7970e2-fc90-49be-a415-167a86fe468b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:42.150337 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-ce7970e2-fc90-49be-a415-167a86fe468b None None] MFA Rules not processed for user `7a4ea9e3987e4f0b9ad9ebe6d3c69b7b`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:42.151766 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-ce7970e2-fc90-49be-a415-167a86fe468b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:42.204169 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:07:42.249259 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 726/1412] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:07:42 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:07:42.250863 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41ce94f-1b3e-400b-96ef-94898ab6f52d None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:42.251154 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41ce94f-1b3e-400b-96ef-94898ab6f52d None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:42.251433 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41ce94f-1b3e-400b-96ef-94898ab6f52d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:42.262054 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255e6bcb-5ad1-4be5-b81c-65938c2a74ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:42.262748 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255e6bcb-5ad1-4be5-b81c-65938c2a74ef admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:42.263034 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255e6bcb-5ad1-4be5-b81c-65938c2a74ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:42.263361 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255e6bcb-5ad1-4be5-b81c-65938c2a74ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:42.264208 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255e6bcb-5ad1-4be5-b81c-65938c2a74ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:42.270022 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255e6bcb-5ad1-4be5-b81c-65938c2a74ef admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:42.296133 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 687/1413] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:07:42 2022] GET /identity/v3/roles => generated 1420 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:07:42.296133 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb776a5-d971-4ef1-99d1-a9567bcfd3a8 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:42.296133 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb776a5-d971-4ef1-99d1-a9567bcfd3a8 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:42.296133 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb776a5-d971-4ef1-99d1-a9567bcfd3a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:42.320429 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-a41ce94f-1b3e-400b-96ef-94898ab6f52d None None] MFA Rules not processed for user `ab344e29f747423fb8766f2d8fbfef0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:42.321725 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-a41ce94f-1b3e-400b-96ef-94898ab6f52d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:42.355388 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-0cb776a5-d971-4ef1-99d1-a9567bcfd3a8 None None] MFA Rules not processed for user `7a4ea9e3987e4f0b9ad9ebe6d3c69b7b`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:42.356838 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-0cb776a5-d971-4ef1-99d1-a9567bcfd3a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:42.375730 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 688/1414] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:07:42 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:07:42.376648 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb776a5-d971-4ef1-99d1-a9567bcfd3a8 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:42.402661 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb776a5-d971-4ef1-99d1-a9567bcfd3a8 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:42.429234 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27171dca-f1c0-4dbe-9ce5-ca7dc3ed3efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:42.432976 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27171dca-f1c0-4dbe-9ce5-ca7dc3ed3efd admin admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:42.434058 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27171dca-f1c0-4dbe-9ce5-ca7dc3ed3efd admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:42.434434 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27171dca-f1c0-4dbe-9ce5-ca7dc3ed3efd admin admin] PATH_INFO: `/v3/projects/7318cb48c37b44a9bcd1f857f936ecc1/users/9ed6a5af303b4142b82bc6333dcceac9/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:42.436092 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:07:42.441791 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27171dca-f1c0-4dbe-9ce5-ca7dc3ed3efd admin admin] RBAC: Authorizing `identity:create_grant(project_id=7318cb48c37b44a9bcd1f857f936ecc1, user_id=9ed6a5af303b4142b82bc6333dcceac9, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:42.447291 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27171dca-f1c0-4dbe-9ce5-ca7dc3ed3efd admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:42.476132 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 689/1415] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:07:42 2022] PUT /identity/v3/projects/7318cb48c37b44a9bcd1f857f936ecc1/users/9ed6a5af303b4142b82bc6333dcceac9/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:07:42.477426 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27171dca-f1c0-4dbe-9ce5-ca7dc3ed3efd admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:42.488927 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-27171dca-f1c0-4dbe-9ce5-ca7dc3ed3efd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:42.537544 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 727/1416] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:07:42 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:07:42.548812 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41446be6-289a-412c-a847-ae73f1112258 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:42.549406 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41446be6-289a-412c-a847-ae73f1112258 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:42.549691 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41446be6-289a-412c-a847-ae73f1112258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:42.659026 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-41446be6-289a-412c-a847-ae73f1112258 None None] MFA Rules not processed for user `9ed6a5af303b4142b82bc6333dcceac9`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:42.665150 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-41446be6-289a-412c-a847-ae73f1112258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:42.689341 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27171dca-f1c0-4dbe-9ce5-ca7dc3ed3efd admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:42.696721 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff66f7b-a5f4-4afb-8e1e-d8e32dddcc42 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b07bf0faca47b28b50990d1720853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:42.697654 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff66f7b-a5f4-4afb-8e1e-d8e32dddcc42 service nova] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:42.698227 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff66f7b-a5f4-4afb-8e1e-d8e32dddcc42 service nova] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:42.698723 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff66f7b-a5f4-4afb-8e1e-d8e32dddcc42 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:42.836486 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 728/1417] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:07:42 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:07:42.853808 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41446be6-289a-412c-a847-ae73f1112258 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:42.853888 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff66f7b-a5f4-4afb-8e1e-d8e32dddcc42 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:42.853888 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff66f7b-a5f4-4afb-8e1e-d8e32dddcc42 service nova] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:42.863558 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41446be6-289a-412c-a847-ae73f1112258 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:42.882886 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dde0c23-87e4-45d4-97cf-d26f382f529b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b07bf0faca47b28b50990d1720853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:42.883657 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dde0c23-87e4-45d4-97cf-d26f382f529b service nova] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:42.885073 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dde0c23-87e4-45d4-97cf-d26f382f529b service nova] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:42.886499 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dde0c23-87e4-45d4-97cf-d26f382f529b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:42.904217 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 690/1418] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:07:42 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:07:42.905688 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff66f7b-a5f4-4afb-8e1e-d8e32dddcc42 service nova] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:42.913852 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-6ff66f7b-a5f4-4afb-8e1e-d8e32dddcc42 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:43.054961 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dde0c23-87e4-45d4-97cf-d26f382f529b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:43.056645 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dde0c23-87e4-45d4-97cf-d26f382f529b service nova] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:43.089802 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 729/1419] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:07:42 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:07:43.104667 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:07:43.136729 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff66f7b-a5f4-4afb-8e1e-d8e32dddcc42 service nova] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:43.151665 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa08d03-a444-4c32-865f-3f4959608e56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab599d98b0624a2e89090312bcab713f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:43.156116 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa08d03-a444-4c32-865f-3f4959608e56 service cinder] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:43.156116 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa08d03-a444-4c32-865f-3f4959608e56 service cinder] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:43.156116 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa08d03-a444-4c32-865f-3f4959608e56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:43.183852 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa08d03-a444-4c32-865f-3f4959608e56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:43.185943 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa08d03-a444-4c32-865f-3f4959608e56 service cinder] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:43.211370 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 691/1420] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:07:42 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:07:43.770307 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dde0c23-87e4-45d4-97cf-d26f382f529b service nova] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:44.016685 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dde0c23-87e4-45d4-97cf-d26f382f529b service nova] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:44.019454 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:07:44.032877 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e813a5-db7e-4365-97aa-8fe2ae76b724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:44.033732 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e813a5-db7e-4365-97aa-8fe2ae76b724 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:44.035515 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e813a5-db7e-4365-97aa-8fe2ae76b724 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:44.036312 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e813a5-db7e-4365-97aa-8fe2ae76b724 admin admin] PATH_INFO: `/v3/users/c357d22167894543ad8e9648ea7894d7` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:44.038785 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e813a5-db7e-4365-97aa-8fe2ae76b724 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c357d22167894543ad8e9648ea7894d7)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:44.042788 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e813a5-db7e-4365-97aa-8fe2ae76b724 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:44.175138 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bdac30-88af-4773-bcef-251ce8529333 None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:44.175661 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bdac30-88af-4773-bcef-251ce8529333 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:44.175771 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bdac30-88af-4773-bcef-251ce8529333 None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:44.177608 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 692/1421] 10.222.0.39 () {54 vars in 912 bytes} [Fri Oct 14 03:07:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 03:07:44.186499 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f99b818-0c41-4952-8c46-96536717d5a9 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:44.187055 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f99b818-0c41-4952-8c46-96536717d5a9 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:44.187812 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f99b818-0c41-4952-8c46-96536717d5a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:44.303890 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-5f99b818-0c41-4952-8c46-96536717d5a9 None None] MFA Rules not processed for user `b123c412ac714b06929a4bd86eca71e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:44.305556 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-5f99b818-0c41-4952-8c46-96536717d5a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:44.348874 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-f8e813a5-db7e-4365-97aa-8fe2ae76b724 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c357d22167894543ad8e9648ea7894d7'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:44.351405 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-f8e813a5-db7e-4365-97aa-8fe2ae76b724 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c357d22167894543ad8e9648ea7894d7'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:44.440081 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-f8e813a5-db7e-4365-97aa-8fe2ae76b724 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c357d22167894543ad8e9648ea7894d7'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:44.440780 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-f8e813a5-db7e-4365-97aa-8fe2ae76b724 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c357d22167894543ad8e9648ea7894d7'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:44.441085 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 693/1422] 10.222.0.39 () {58 vars in 1009 bytes} [Fri Oct 14 03:07:44 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 03:07:44.457917 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-f8e813a5-db7e-4365-97aa-8fe2ae76b724 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c357d22167894543ad8e9648ea7894d7'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:44.491312 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f99b818-0c41-4952-8c46-96536717d5a9 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:44.492834 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-5f99b818-0c41-4952-8c46-96536717d5a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:44.501501 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 730/1423] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:07:43 2022] DELETE /identity/v3/users/c357d22167894543ad8e9648ea7894d7 => generated 0 bytes in 735 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:07:44.505702 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06281b3b-f4e8-461c-80e6-35b39268ae78 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:44.506020 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06281b3b-f4e8-461c-80e6-35b39268ae78 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:44.510868 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06281b3b-f4e8-461c-80e6-35b39268ae78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:44.515983 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f99b818-0c41-4952-8c46-96536717d5a9 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:44.550309 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-06281b3b-f4e8-461c-80e6-35b39268ae78 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:44.553746 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-06281b3b-f4e8-461c-80e6-35b39268ae78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:44.726448 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 731/1424] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:07:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:07:44.739750 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1ae4c3-bc08-4f6a-98c7-6cf91a531348 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:44.745604 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1ae4c3-bc08-4f6a-98c7-6cf91a531348 service neutron] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:44.746013 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1ae4c3-bc08-4f6a-98c7-6cf91a531348 service neutron] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:44.746431 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1ae4c3-bc08-4f6a-98c7-6cf91a531348 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:44.766293 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06281b3b-f4e8-461c-80e6-35b39268ae78 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:44.771087 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1ae4c3-bc08-4f6a-98c7-6cf91a531348 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:44.774540 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1ae4c3-bc08-4f6a-98c7-6cf91a531348 service neutron] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:44.782147 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06281b3b-f4e8-461c-80e6-35b39268ae78 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:44.793342 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 694/1425] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:07:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 03:07:44.800112 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47da835-d420-4eba-9729-a2fc7ed6b504 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:44.801206 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47da835-d420-4eba-9729-a2fc7ed6b504 service neutron] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:44.801206 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47da835-d420-4eba-9729-a2fc7ed6b504 service neutron] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:44.801793 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47da835-d420-4eba-9729-a2fc7ed6b504 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:44.827473 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47da835-d420-4eba-9729-a2fc7ed6b504 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:44.830547 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47da835-d420-4eba-9729-a2fc7ed6b504 service neutron] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:44.859418 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 732/1426] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:07:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 03:07:45.093983 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1ae4c3-bc08-4f6a-98c7-6cf91a531348 service neutron] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:45.098428 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-ef1ae4c3-bc08-4f6a-98c7-6cf91a531348 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:45.129608 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1ae4c3-bc08-4f6a-98c7-6cf91a531348 service neutron] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:45.141870 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35a6eec-0b84-4a02-855d-80f4c6f46539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:45.142701 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35a6eec-0b84-4a02-855d-80f4c6f46539 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:45.143079 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35a6eec-0b84-4a02-855d-80f4c6f46539 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:45.144105 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35a6eec-0b84-4a02-855d-80f4c6f46539 admin admin] PATH_INFO: `/v3/users/c3cd74e2ed67443abef1d66a2779a083` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:45.145569 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35a6eec-0b84-4a02-855d-80f4c6f46539 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3cd74e2ed67443abef1d66a2779a083)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:45.147884 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35a6eec-0b84-4a02-855d-80f4c6f46539 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:45.451768 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-e35a6eec-0b84-4a02-855d-80f4c6f46539 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3cd74e2ed67443abef1d66a2779a083'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:45.480224 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-e35a6eec-0b84-4a02-855d-80f4c6f46539 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3cd74e2ed67443abef1d66a2779a083'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:45.513799 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-e35a6eec-0b84-4a02-855d-80f4c6f46539 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3cd74e2ed67443abef1d66a2779a083'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:45.514934 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-e35a6eec-0b84-4a02-855d-80f4c6f46539 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3cd74e2ed67443abef1d66a2779a083'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:45.584434 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-e35a6eec-0b84-4a02-855d-80f4c6f46539 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3cd74e2ed67443abef1d66a2779a083'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:45.599329 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 695/1427] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:07:45 2022] DELETE /identity/v3/users/c3cd74e2ed67443abef1d66a2779a083 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:07:45.601499 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c304772e-e2c7-4069-8a22-c328563c6c7a None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:45.602419 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c304772e-e2c7-4069-8a22-c328563c6c7a None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:45.602871 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c304772e-e2c7-4069-8a22-c328563c6c7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:45.647745 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-c304772e-e2c7-4069-8a22-c328563c6c7a None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:45.651510 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-c304772e-e2c7-4069-8a22-c328563c6c7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:45.810720 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 733/1428] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:07:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:07:45.829393 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5313abdd-f2c1-4234-8782-bf26831f1fbe None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:45.829687 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5313abdd-f2c1-4234-8782-bf26831f1fbe None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:45.830128 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5313abdd-f2c1-4234-8782-bf26831f1fbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:45.839181 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:07:45.897543 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-5313abdd-f2c1-4234-8782-bf26831f1fbe None None] MFA Rules not processed for user `fb1c279c2f7f404ab9c846a3ba594886`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:45.898827 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-5313abdd-f2c1-4234-8782-bf26831f1fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:45.901850 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c304772e-e2c7-4069-8a22-c328563c6c7a None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:45.904015 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-c304772e-e2c7-4069-8a22-c328563c6c7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:45.912601 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:07:46.092178 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c304772e-e2c7-4069-8a22-c328563c6c7a None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:46.093290 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 696/1429] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:07:45 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:07:46.094290 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5313abdd-f2c1-4234-8782-bf26831f1fbe None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:46.104167 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-5313abdd-f2c1-4234-8782-bf26831f1fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:46.113750 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934ef6c1-195b-4a05-8453-fc6ec842c2b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:46.114506 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934ef6c1-195b-4a05-8453-fc6ec842c2b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:46.114812 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934ef6c1-195b-4a05-8453-fc6ec842c2b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:46.115062 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934ef6c1-195b-4a05-8453-fc6ec842c2b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:46.143290 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934ef6c1-195b-4a05-8453-fc6ec842c2b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:46.145452 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5313abdd-f2c1-4234-8782-bf26831f1fbe None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:46.145703 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934ef6c1-195b-4a05-8453-fc6ec842c2b8 service neutron] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:46.161875 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379e3c15-7b0b-44a2-8339-f95677ad59da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:46.162542 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e3c15-7b0b-44a2-8339-f95677ad59da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:46.162829 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e3c15-7b0b-44a2-8339-f95677ad59da admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:46.163111 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e3c15-7b0b-44a2-8339-f95677ad59da admin admin] PATH_INFO: `/v3/users/07afc8012c964bf39039dcf26ee38186` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:46.168730 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379e3c15-7b0b-44a2-8339-f95677ad59da admin admin] RBAC: Authorizing `identity:delete_user(user_id=07afc8012c964bf39039dcf26ee38186)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:46.170767 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379e3c15-7b0b-44a2-8339-f95677ad59da admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:46.184316 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 734/1430] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:07:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:07:46.185241 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934ef6c1-195b-4a05-8453-fc6ec842c2b8 service neutron] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:46.190973 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-934ef6c1-195b-4a05-8453-fc6ec842c2b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:46.369616 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-379e3c15-7b0b-44a2-8339-f95677ad59da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07afc8012c964bf39039dcf26ee38186'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:46.386153 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-379e3c15-7b0b-44a2-8339-f95677ad59da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07afc8012c964bf39039dcf26ee38186'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:46.401500 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-379e3c15-7b0b-44a2-8339-f95677ad59da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07afc8012c964bf39039dcf26ee38186'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:46.402663 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-379e3c15-7b0b-44a2-8339-f95677ad59da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07afc8012c964bf39039dcf26ee38186'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:46.406983 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934ef6c1-195b-4a05-8453-fc6ec842c2b8 service neutron] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:46.418372 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed04b8bd-9a40-41ae-8020-bdb115c5e1e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b07bf0faca47b28b50990d1720853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:46.419007 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed04b8bd-9a40-41ae-8020-bdb115c5e1e3 service nova] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:46.419292 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed04b8bd-9a40-41ae-8020-bdb115c5e1e3 service nova] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:46.419569 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed04b8bd-9a40-41ae-8020-bdb115c5e1e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:46.421407 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-ed04b8bd-9a40-41ae-8020-bdb115c5e1e3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:46.442661 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed04b8bd-9a40-41ae-8020-bdb115c5e1e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:46.445037 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed04b8bd-9a40-41ae-8020-bdb115c5e1e3 service nova] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:46.503228 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-379e3c15-7b0b-44a2-8339-f95677ad59da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07afc8012c964bf39039dcf26ee38186'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:46.508468 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 735/1431] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:07:46 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:07:46.516314 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 697/1432] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:07:46 2022] DELETE /identity/v3/users/07afc8012c964bf39039dcf26ee38186 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:07:46.517937 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be5fe7d-c262-4f61-bebb-15a970ca9b56 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:46.518280 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be5fe7d-c262-4f61-bebb-15a970ca9b56 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:46.518567 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be5fe7d-c262-4f61-bebb-15a970ca9b56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:46.522309 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed04b8bd-9a40-41ae-8020-bdb115c5e1e3 service nova] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:46.530109 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-ed04b8bd-9a40-41ae-8020-bdb115c5e1e3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:46.608033 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed04b8bd-9a40-41ae-8020-bdb115c5e1e3 service nova] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:46.627085 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-4be5fe7d-c262-4f61-bebb-15a970ca9b56 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:46.632039 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-4be5fe7d-c262-4f61-bebb-15a970ca9b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:46.778879 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 698/1433] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:07:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:07:46.780069 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be5fe7d-c262-4f61-bebb-15a970ca9b56 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:46.787758 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-4be5fe7d-c262-4f61-bebb-15a970ca9b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:46.839006 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c11a493-6959-4930-9974-3a9fe47b333f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:46.839912 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c11a493-6959-4930-9974-3a9fe47b333f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:46.840387 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c11a493-6959-4930-9974-3a9fe47b333f admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:46.840954 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c11a493-6959-4930-9974-3a9fe47b333f admin admin] PATH_INFO: `/v3/projects/872d030527f5428489100defa400a9ba` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:46.842319 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c11a493-6959-4930-9974-3a9fe47b333f admin admin] RBAC: Authorizing `identity:delete_project(project_id=872d030527f5428489100defa400a9ba)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:46.845475 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c11a493-6959-4930-9974-3a9fe47b333f admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:46.924524 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be5fe7d-c262-4f61-bebb-15a970ca9b56 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:46.930823 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b006b74-00d6-4081-925c-e13c7ee4bd6f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '922ef7811aed450baff7f936011b0af1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:46.931985 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b006b74-00d6-4081-925c-e13c7ee4bd6f service placement] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:46.931985 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b006b74-00d6-4081-925c-e13c7ee4bd6f service placement] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:46.932150 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b006b74-00d6-4081-925c-e13c7ee4bd6f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:46.934210 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-6b006b74-00d6-4081-925c-e13c7ee4bd6f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:46.956409 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b006b74-00d6-4081-925c-e13c7ee4bd6f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:46.958200 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b006b74-00d6-4081-925c-e13c7ee4bd6f service placement] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:46.977799 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 699/1434] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:07:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:07:46.980697 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce405946-96d6-4a85-9fc3-1fee864b67c3 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:46.980985 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce405946-96d6-4a85-9fc3-1fee864b67c3 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:46.992067 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce405946-96d6-4a85-9fc3-1fee864b67c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:47.030059 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-2c11a493-6959-4930-9974-3a9fe47b333f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '872d030527f5428489100defa400a9ba', 'request_id': 'req-2c11a493-6959-4930-9974-3a9fe47b333f'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:47.037332 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-2c11a493-6959-4930-9974-3a9fe47b333f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '872d030527f5428489100defa400a9ba', 'request_id': 'req-2c11a493-6959-4930-9974-3a9fe47b333f'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:47.080555 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-ce405946-96d6-4a85-9fc3-1fee864b67c3 None None] MFA Rules not processed for user `7a4ea9e3987e4f0b9ad9ebe6d3c69b7b`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:47.081488 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-ce405946-96d6-4a85-9fc3-1fee864b67c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:47.119581 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-2c11a493-6959-4930-9974-3a9fe47b333f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '872d030527f5428489100defa400a9ba', 'request_id': 'req-2c11a493-6959-4930-9974-3a9fe47b333f'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:47.150134 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-2c11a493-6959-4930-9974-3a9fe47b333f admin admin] The token cache is being invalidate because project 872d030527f5428489100defa400a9ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81517) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:07:47.151352 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-2c11a493-6959-4930-9974-3a9fe47b333f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:47.153319 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-2c11a493-6959-4930-9974-3a9fe47b333f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:47.155589 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 736/1435] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:07:46 2022] DELETE /identity/v3/projects/872d030527f5428489100defa400a9ba => generated 0 bytes in 648 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:07:47.159708 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c11a493-6959-4930-9974-3a9fe47b333f admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:47.162500 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-2c11a493-6959-4930-9974-3a9fe47b333f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:47.209278 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 700/1436] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:07:46 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:07:47.210961 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f333c431-3072-4ed4-b497-29d75ad247af None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:47.211402 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f333c431-3072-4ed4-b497-29d75ad247af None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:47.212111 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f333c431-3072-4ed4-b497-29d75ad247af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:47.253239 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:07:47.303439 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-f333c431-3072-4ed4-b497-29d75ad247af None None] MFA Rules not processed for user `7517af640df3402783a38befbf3d3908`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:47.304714 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-f333c431-3072-4ed4-b497-29d75ad247af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:47.331689 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c11a493-6959-4930-9974-3a9fe47b333f admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:47.337483 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428e1703-a8c9-4d73-abf2-d1ebdff71010 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:47.338195 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428e1703-a8c9-4d73-abf2-d1ebdff71010 service neutron] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:47.338483 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428e1703-a8c9-4d73-abf2-d1ebdff71010 service neutron] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:47.338761 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428e1703-a8c9-4d73-abf2-d1ebdff71010 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:47.342400 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-428e1703-a8c9-4d73-abf2-d1ebdff71010 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:47.431955 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:07:47.469986 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 701/1437] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:07:47 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:07:47.487576 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f333c431-3072-4ed4-b497-29d75ad247af None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:47.490981 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-f333c431-3072-4ed4-b497-29d75ad247af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:47.502898 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428e1703-a8c9-4d73-abf2-d1ebdff71010 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:47.504919 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428e1703-a8c9-4d73-abf2-d1ebdff71010 service neutron] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:47.521011 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 737/1438] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:07:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:07:47.522477 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b6b493-6136-4032-a3b3-45033fa4e5b2 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:47.522768 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b6b493-6136-4032-a3b3-45033fa4e5b2 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:47.523043 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b6b493-6136-4032-a3b3-45033fa4e5b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:47.544774 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f333c431-3072-4ed4-b497-29d75ad247af None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:47.560821 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72ffd4c-ea17-472f-8a85-6613b2276766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:47.564264 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72ffd4c-ea17-472f-8a85-6613b2276766 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:47.564734 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72ffd4c-ea17-472f-8a85-6613b2276766 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:47.565161 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72ffd4c-ea17-472f-8a85-6613b2276766 admin admin] PATH_INFO: `/v3/users/6b05fa1cc14b4e7db2f19d0a24b98be0` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:47.570684 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72ffd4c-ea17-472f-8a85-6613b2276766 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b05fa1cc14b4e7db2f19d0a24b98be0)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:47.573687 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72ffd4c-ea17-472f-8a85-6613b2276766 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:47.584171 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-49b6b493-6136-4032-a3b3-45033fa4e5b2 None None] MFA Rules not processed for user `7517af640df3402783a38befbf3d3908`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:47.586406 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-49b6b493-6136-4032-a3b3-45033fa4e5b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:47.606739 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 738/1439] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:07:47 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:07:47.622257 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b6b493-6136-4032-a3b3-45033fa4e5b2 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:47.627398 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-49b6b493-6136-4032-a3b3-45033fa4e5b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:47.806216 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-a72ffd4c-ea17-472f-8a85-6613b2276766 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b05fa1cc14b4e7db2f19d0a24b98be0'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:47.824470 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-a72ffd4c-ea17-472f-8a85-6613b2276766 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b05fa1cc14b4e7db2f19d0a24b98be0'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:47.827187 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b6b493-6136-4032-a3b3-45033fa4e5b2 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:47.841396 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e148b36f-9731-418d-94a2-d3b88c5ca2a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b07bf0faca47b28b50990d1720853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:47.842423 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e148b36f-9731-418d-94a2-d3b88c5ca2a6 service nova] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:47.842713 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e148b36f-9731-418d-94a2-d3b88c5ca2a6 service nova] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:47.842990 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e148b36f-9731-418d-94a2-d3b88c5ca2a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:47.854416 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-a72ffd4c-ea17-472f-8a85-6613b2276766 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b05fa1cc14b4e7db2f19d0a24b98be0'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:47.857742 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-a72ffd4c-ea17-472f-8a85-6613b2276766 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b05fa1cc14b4e7db2f19d0a24b98be0'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:47.868038 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e148b36f-9731-418d-94a2-d3b88c5ca2a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:47.870218 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e148b36f-9731-418d-94a2-d3b88c5ca2a6 service nova] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:47.882240 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 739/1440] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:07:47 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:07:47.943906 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-a72ffd4c-ea17-472f-8a85-6613b2276766 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b05fa1cc14b4e7db2f19d0a24b98be0'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:47.952967 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 702/1441] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:07:47 2022] DELETE /identity/v3/users/6b05fa1cc14b4e7db2f19d0a24b98be0 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:07:48.053293 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:07:48.223506 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:07:48.375859 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e148b36f-9731-418d-94a2-d3b88c5ca2a6 service nova] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:48.377798 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-e148b36f-9731-418d-94a2-d3b88c5ca2a6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:48.402646 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e148b36f-9731-418d-94a2-d3b88c5ca2a6 service nova] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:48.424707 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72ffd4c-ea17-472f-8a85-6613b2276766 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:48.426188 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-a72ffd4c-ea17-472f-8a85-6613b2276766 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:48.527604 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72ffd4c-ea17-472f-8a85-6613b2276766 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:48.533610 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aba0298-10fd-4b20-98ba-d81365fb7a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:48.536153 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aba0298-10fd-4b20-98ba-d81365fb7a03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:48.537009 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aba0298-10fd-4b20-98ba-d81365fb7a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:48.537651 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aba0298-10fd-4b20-98ba-d81365fb7a03 admin admin] PATH_INFO: `/v3/projects/81ac2ef8804b47d785f51e605731ad5b` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:48.541030 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aba0298-10fd-4b20-98ba-d81365fb7a03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81ac2ef8804b47d785f51e605731ad5b)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:48.546538 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aba0298-10fd-4b20-98ba-d81365fb7a03 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:48.551427 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c54c05-8eef-42d2-a83d-14dabb36e275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:48.552121 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c54c05-8eef-42d2-a83d-14dabb36e275 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:48.552407 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c54c05-8eef-42d2-a83d-14dabb36e275 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:48.552682 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c54c05-8eef-42d2-a83d-14dabb36e275 admin admin] PATH_INFO: `/v3/projects/f7140b54e63c4d5e8df904f73c0192c2` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:48.558571 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c54c05-8eef-42d2-a83d-14dabb36e275 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7140b54e63c4d5e8df904f73c0192c2)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:48.560567 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c54c05-8eef-42d2-a83d-14dabb36e275 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:48.711221 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-9aba0298-10fd-4b20-98ba-d81365fb7a03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81ac2ef8804b47d785f51e605731ad5b', 'request_id': 'req-9aba0298-10fd-4b20-98ba-d81365fb7a03'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:48.723855 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-38c54c05-8eef-42d2-a83d-14dabb36e275 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7140b54e63c4d5e8df904f73c0192c2', 'request_id': 'req-38c54c05-8eef-42d2-a83d-14dabb36e275'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:48.731001 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-9aba0298-10fd-4b20-98ba-d81365fb7a03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81ac2ef8804b47d785f51e605731ad5b', 'request_id': 'req-9aba0298-10fd-4b20-98ba-d81365fb7a03'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:48.740196 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-38c54c05-8eef-42d2-a83d-14dabb36e275 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7140b54e63c4d5e8df904f73c0192c2', 'request_id': 'req-38c54c05-8eef-42d2-a83d-14dabb36e275'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:48.857575 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-38c54c05-8eef-42d2-a83d-14dabb36e275 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7140b54e63c4d5e8df904f73c0192c2', 'request_id': 'req-38c54c05-8eef-42d2-a83d-14dabb36e275'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:48.886089 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-38c54c05-8eef-42d2-a83d-14dabb36e275 admin admin] The token cache is being invalidate because project f7140b54e63c4d5e8df904f73c0192c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81517) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:07:48.886936 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-38c54c05-8eef-42d2-a83d-14dabb36e275 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:48.888186 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-38c54c05-8eef-42d2-a83d-14dabb36e275 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:48.893610 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 740/1442] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:07:48 2022] DELETE /identity/v3/projects/f7140b54e63c4d5e8df904f73c0192c2 => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:07:48.897645 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef078ce-8aa6-4232-9087-02b4d1bcfa7a None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:48.898612 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef078ce-8aa6-4232-9087-02b4d1bcfa7a None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:48.902601 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef078ce-8aa6-4232-9087-02b4d1bcfa7a None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:48.902601 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 741/1443] 10.222.0.39 () {54 vars in 912 bytes} [Fri Oct 14 03:07:48 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 03:07:48.907337 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef078ce-8aa6-4232-9087-02b4d1bcfa7a None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:48.913920 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-2ef078ce-8aa6-4232-9087-02b4d1bcfa7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:48.950546 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-9aba0298-10fd-4b20-98ba-d81365fb7a03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81ac2ef8804b47d785f51e605731ad5b', 'request_id': 'req-9aba0298-10fd-4b20-98ba-d81365fb7a03'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:48.979827 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-9aba0298-10fd-4b20-98ba-d81365fb7a03 admin admin] The token cache is being invalidate because project 81ac2ef8804b47d785f51e605731ad5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:07:48.980452 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-9aba0298-10fd-4b20-98ba-d81365fb7a03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:48.983169 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-9aba0298-10fd-4b20-98ba-d81365fb7a03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:48.985793 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 703/1444] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:07:48 2022] DELETE /identity/v3/projects/81ac2ef8804b47d785f51e605731ad5b => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:07:48.993034 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d67bc7-f47e-4dcc-a38f-bda2f5e80fd3 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:48.993655 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d67bc7-f47e-4dcc-a38f-bda2f5e80fd3 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:48.995613 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d67bc7-f47e-4dcc-a38f-bda2f5e80fd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:49.039250 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-42d67bc7-f47e-4dcc-a38f-bda2f5e80fd3 None None] MFA Rules not processed for user `b123c412ac714b06929a4bd86eca71e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:49.041975 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-42d67bc7-f47e-4dcc-a38f-bda2f5e80fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:49.189991 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:07:49.204697 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 704/1445] 10.222.0.39 () {58 vars in 1009 bytes} [Fri Oct 14 03:07:48 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:07:49.217102 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d67bc7-f47e-4dcc-a38f-bda2f5e80fd3 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:49.220624 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-42d67bc7-f47e-4dcc-a38f-bda2f5e80fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:49.224705 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef078ce-8aa6-4232-9087-02b4d1bcfa7a None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:49.243069 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d67bc7-f47e-4dcc-a38f-bda2f5e80fd3 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:49.246552 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd3dedd-a821-4a9d-b746-17addc787fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:49.248392 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd3dedd-a821-4a9d-b746-17addc787fd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:49.248672 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd3dedd-a821-4a9d-b746-17addc787fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:49.248950 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd3dedd-a821-4a9d-b746-17addc787fd9 admin admin] PATH_INFO: `/v3/users/4c3c26f22bb548ce99075c49c79443a4` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:49.252137 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e632d5-f58f-4aa9-888f-589348594464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:49.252886 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e632d5-f58f-4aa9-888f-589348594464 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:49.253226 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e632d5-f58f-4aa9-888f-589348594464 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:49.253651 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e632d5-f58f-4aa9-888f-589348594464 admin admin] PATH_INFO: `/v3/users/9ed6a5af303b4142b82bc6333dcceac9` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:49.265338 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd3dedd-a821-4a9d-b746-17addc787fd9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c3c26f22bb548ce99075c49c79443a4)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:49.265931 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e632d5-f58f-4aa9-888f-589348594464 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ed6a5af303b4142b82bc6333dcceac9)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:49.265931 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e632d5-f58f-4aa9-888f-589348594464 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:49.266055 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd3dedd-a821-4a9d-b746-17addc787fd9 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:49.529078 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-e2e632d5-f58f-4aa9-888f-589348594464 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ed6a5af303b4142b82bc6333dcceac9'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:49.541999 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-2cd3dedd-a821-4a9d-b746-17addc787fd9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c3c26f22bb548ce99075c49c79443a4'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:49.545125 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-e2e632d5-f58f-4aa9-888f-589348594464 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ed6a5af303b4142b82bc6333dcceac9'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:49.545284 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-2cd3dedd-a821-4a9d-b746-17addc787fd9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c3c26f22bb548ce99075c49c79443a4'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:49.583445 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:07:49.584321 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-e2e632d5-f58f-4aa9-888f-589348594464 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ed6a5af303b4142b82bc6333dcceac9'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:49.585988 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-e2e632d5-f58f-4aa9-888f-589348594464 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ed6a5af303b4142b82bc6333dcceac9'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:49.640479 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-2cd3dedd-a821-4a9d-b746-17addc787fd9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c3c26f22bb548ce99075c49c79443a4'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:49.641124 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-2cd3dedd-a821-4a9d-b746-17addc787fd9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c3c26f22bb548ce99075c49c79443a4'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:49.658618 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-2cd3dedd-a821-4a9d-b746-17addc787fd9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c3c26f22bb548ce99075c49c79443a4'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:49.706201 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 742/1446] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:07:48 2022] DELETE /identity/v3/users/4c3c26f22bb548ce99075c49c79443a4 => generated 0 bytes in 804 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:07:49.707668 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd3dedd-a821-4a9d-b746-17addc787fd9 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:49.710322 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-2cd3dedd-a821-4a9d-b746-17addc787fd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:49.755031 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-e2e632d5-f58f-4aa9-888f-589348594464 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ed6a5af303b4142b82bc6333dcceac9'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:49.766945 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 705/1447] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:07:49 2022] DELETE /identity/v3/users/9ed6a5af303b4142b82bc6333dcceac9 => generated 0 bytes in 557 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:07:49.767948 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e632d5-f58f-4aa9-888f-589348594464 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:49.771206 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-e2e632d5-f58f-4aa9-888f-589348594464 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:49.924407 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd3dedd-a821-4a9d-b746-17addc787fd9 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:50.021069 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e632d5-f58f-4aa9-888f-589348594464 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:50.041501 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82d5552-932d-4f81-b390-0b064d23a081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:50.042188 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82d5552-932d-4f81-b390-0b064d23a081 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:50.043296 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82d5552-932d-4f81-b390-0b064d23a081 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:50.043616 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82d5552-932d-4f81-b390-0b064d23a081 admin admin] PATH_INFO: `/v3/projects/e9eae27dea844c47841f2a16a4325ab8` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:50.052673 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82d5552-932d-4f81-b390-0b064d23a081 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9eae27dea844c47841f2a16a4325ab8)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:50.057619 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82d5552-932d-4f81-b390-0b064d23a081 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:50.116199 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e8b5fc-a991-4a87-a9d2-a7c9cb7b35c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:50.116897 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e8b5fc-a991-4a87-a9d2-a7c9cb7b35c3 service neutron] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:50.117182 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e8b5fc-a991-4a87-a9d2-a7c9cb7b35c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:50.117455 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e8b5fc-a991-4a87-a9d2-a7c9cb7b35c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:50.126797 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-88e8b5fc-a991-4a87-a9d2-a7c9cb7b35c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:50.145320 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e8b5fc-a991-4a87-a9d2-a7c9cb7b35c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:50.147160 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e8b5fc-a991-4a87-a9d2-a7c9cb7b35c3 service neutron] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:50.174575 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 743/1448] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:07:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:07:50.311503 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-c82d5552-932d-4f81-b390-0b064d23a081 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9eae27dea844c47841f2a16a4325ab8', 'request_id': 'req-c82d5552-932d-4f81-b390-0b064d23a081'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:50.325427 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-c82d5552-932d-4f81-b390-0b064d23a081 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9eae27dea844c47841f2a16a4325ab8', 'request_id': 'req-c82d5552-932d-4f81-b390-0b064d23a081'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:50.417889 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-c82d5552-932d-4f81-b390-0b064d23a081 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9eae27dea844c47841f2a16a4325ab8', 'request_id': 'req-c82d5552-932d-4f81-b390-0b064d23a081'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:50.437088 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-c82d5552-932d-4f81-b390-0b064d23a081 admin admin] The token cache is being invalidate because project e9eae27dea844c47841f2a16a4325ab8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:07:50.437394 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-c82d5552-932d-4f81-b390-0b064d23a081 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:50.438119 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-c82d5552-932d-4f81-b390-0b064d23a081 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:50.440103 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 706/1449] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:07:49 2022] DELETE /identity/v3/projects/e9eae27dea844c47841f2a16a4325ab8 => generated 0 bytes in 672 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:07:50.444875 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81585068-613a-409e-9865-2076bde092ba None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:50.445720 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81585068-613a-409e-9865-2076bde092ba None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:50.446210 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81585068-613a-409e-9865-2076bde092ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:50.492223 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-81585068-613a-409e-9865-2076bde092ba None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:50.495027 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-81585068-613a-409e-9865-2076bde092ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:50.674694 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 744/1450] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:07:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:07:50.684896 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee90bc4b-a640-4790-92ca-1ec79c6323a1 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:50.686553 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee90bc4b-a640-4790-92ca-1ec79c6323a1 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:50.687146 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee90bc4b-a640-4790-92ca-1ec79c6323a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:50.727700 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-ee90bc4b-a640-4790-92ca-1ec79c6323a1 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:50.729689 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-ee90bc4b-a640-4790-92ca-1ec79c6323a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:50.755513 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 707/1451] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:07:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:07:50.759728 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81585068-613a-409e-9865-2076bde092ba None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:50.782215 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81585068-613a-409e-9865-2076bde092ba None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:50.796840 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e658ddd-c7dd-4bcf-b332-f74bf8f3fcaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:50.797574 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e658ddd-c7dd-4bcf-b332-f74bf8f3fcaa admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:50.797955 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e658ddd-c7dd-4bcf-b332-f74bf8f3fcaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:50.798355 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e658ddd-c7dd-4bcf-b332-f74bf8f3fcaa admin admin] PATH_INFO: `/v3/domains` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:50.799099 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e658ddd-c7dd-4bcf-b332-f74bf8f3fcaa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 03:07:50.800081 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e658ddd-c7dd-4bcf-b332-f74bf8f3fcaa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:50.801888 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e658ddd-c7dd-4bcf-b332-f74bf8f3fcaa admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:50.842509 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee90bc4b-a640-4790-92ca-1ec79c6323a1 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:50.849002 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 745/1452] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Oct 14 03:07:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 03:07:50.854123 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee90bc4b-a640-4790-92ca-1ec79c6323a1 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:50.856419 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:07:50.861866 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2072c70b-5b5f-45c3-87b3-110d029b7efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:50.863553 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2072c70b-5b5f-45c3-87b3-110d029b7efd admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:50.864022 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2072c70b-5b5f-45c3-87b3-110d029b7efd admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:50.864781 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2072c70b-5b5f-45c3-87b3-110d029b7efd admin admin] PATH_INFO: `/v3/projects` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:50.865906 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2072c70b-5b5f-45c3-87b3-110d029b7efd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:50.868112 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2072c70b-5b5f-45c3-87b3-110d029b7efd admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:51.005380 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 708/1453] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:07:50 2022] POST /identity/v3/projects => generated 363 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:07:51.010191 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e658ddd-c7dd-4bcf-b332-f74bf8f3fcaa admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:51.033420 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e658ddd-c7dd-4bcf-b332-f74bf8f3fcaa admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:51.229284 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6838b75-029d-4752-9088-c3c1778b5b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:51.232478 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675df987-a41f-4479-b8ba-eda1db0599ac None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:51.232553 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6838b75-029d-4752-9088-c3c1778b5b76 admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:51.232631 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675df987-a41f-4479-b8ba-eda1db0599ac None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:51.232698 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6838b75-029d-4752-9088-c3c1778b5b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:51.232763 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675df987-a41f-4479-b8ba-eda1db0599ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:51.232826 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6838b75-029d-4752-9088-c3c1778b5b76 admin admin] PATH_INFO: `/v3/users` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:51.232826 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6838b75-029d-4752-9088-c3c1778b5b76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:51.233722 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6838b75-029d-4752-9088-c3c1778b5b76 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:51.314390 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-675df987-a41f-4479-b8ba-eda1db0599ac None None] MFA Rules not processed for user `410b7c9e93c94ce78d1b5c0e646307a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:51.317389 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-675df987-a41f-4479-b8ba-eda1db0599ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:51.332041 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:07:51.417967 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 746/1454] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:07:51 2022] POST /identity/v3/users => generated 282 bytes in 409 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:07:51.419076 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6838b75-029d-4752-9088-c3c1778b5b76 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:51.421542 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-f6838b75-029d-4752-9088-c3c1778b5b76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:51.436177 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 709/1455] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:07:51 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:07:51.437436 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675df987-a41f-4479-b8ba-eda1db0599ac None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:51.456530 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6838b75-029d-4752-9088-c3c1778b5b76 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:51.463605 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675df987-a41f-4479-b8ba-eda1db0599ac None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:51.467014 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6483aee-fc68-4942-aa75-58ed8911797e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:51.467722 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6483aee-fc68-4942-aa75-58ed8911797e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:51.468006 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6483aee-fc68-4942-aa75-58ed8911797e admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:51.468456 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6483aee-fc68-4942-aa75-58ed8911797e admin admin] PATH_INFO: `/v3/projects/d7d00636950b44e7b6a4ef7496bae560` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:51.470699 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6483aee-fc68-4942-aa75-58ed8911797e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7d00636950b44e7b6a4ef7496bae560)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:51.472903 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6483aee-fc68-4942-aa75-58ed8911797e admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:51.473668 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e4bbf7-f93e-4d27-863f-07f8d451cc29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:51.475171 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e4bbf7-f93e-4d27-863f-07f8d451cc29 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:51.475559 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e4bbf7-f93e-4d27-863f-07f8d451cc29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:51.475939 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e4bbf7-f93e-4d27-863f-07f8d451cc29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:51.476705 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e4bbf7-f93e-4d27-863f-07f8d451cc29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:51.478659 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e4bbf7-f93e-4d27-863f-07f8d451cc29 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:51.511531 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 710/1456] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:07:51 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:07:51.517980 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e4bbf7-f93e-4d27-863f-07f8d451cc29 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:51.564363 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e4bbf7-f93e-4d27-863f-07f8d451cc29 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:51.589896 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d7c7ad-71a6-422f-ab24-038865e09528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:51.593462 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d7c7ad-71a6-422f-ab24-038865e09528 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:51.595545 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d7c7ad-71a6-422f-ab24-038865e09528 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:51.595967 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d7c7ad-71a6-422f-ab24-038865e09528 admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:51.596986 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d7c7ad-71a6-422f-ab24-038865e09528 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:51.602343 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d7c7ad-71a6-422f-ab24-038865e09528 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:51.642674 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 711/1457] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:07:51 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:07:51.648538 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d7c7ad-71a6-422f-ab24-038865e09528 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:51.671742 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d7c7ad-71a6-422f-ab24-038865e09528 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:51.684424 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e18b4c-1e6d-4030-9407-e1702b92b848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:51.687593 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e18b4c-1e6d-4030-9407-e1702b92b848 admin admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:51.688609 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e18b4c-1e6d-4030-9407-e1702b92b848 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:51.689137 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e18b4c-1e6d-4030-9407-e1702b92b848 admin admin] PATH_INFO: `/v3/projects/5c47a804d4d9476c868bcbe3b4097442/users/469802150c6b4cb886eee2a33f618e2f/roles/9bd0470b544240209a420f640c79c06c` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:51.722122 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e18b4c-1e6d-4030-9407-e1702b92b848 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c47a804d4d9476c868bcbe3b4097442, user_id=469802150c6b4cb886eee2a33f618e2f, role_id=9bd0470b544240209a420f640c79c06c)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:51.722684 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-e6483aee-fc68-4942-aa75-58ed8911797e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7d00636950b44e7b6a4ef7496bae560', 'request_id': 'req-e6483aee-fc68-4942-aa75-58ed8911797e'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:51.726959 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e18b4c-1e6d-4030-9407-e1702b92b848 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:51.741449 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-e6483aee-fc68-4942-aa75-58ed8911797e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7d00636950b44e7b6a4ef7496bae560', 'request_id': 'req-e6483aee-fc68-4942-aa75-58ed8911797e'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:51.873961 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-e6483aee-fc68-4942-aa75-58ed8911797e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7d00636950b44e7b6a4ef7496bae560', 'request_id': 'req-e6483aee-fc68-4942-aa75-58ed8911797e'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:51.878965 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 712/1458] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:07:51 2022] PUT /identity/v3/projects/5c47a804d4d9476c868bcbe3b4097442/users/469802150c6b4cb886eee2a33f618e2f/roles/9bd0470b544240209a420f640c79c06c => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 03:07:51.894446 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b37289-f166-4094-8e98-97fc0f4477eb None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:51.895745 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b37289-f166-4094-8e98-97fc0f4477eb None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:51.896265 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b37289-f166-4094-8e98-97fc0f4477eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:51.903237 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-e6483aee-fc68-4942-aa75-58ed8911797e admin admin] The token cache is being invalidate because project d7d00636950b44e7b6a4ef7496bae560 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81517) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:07:51.903783 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-e6483aee-fc68-4942-aa75-58ed8911797e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:51.909113 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-e6483aee-fc68-4942-aa75-58ed8911797e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:51.912948 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 747/1459] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:07:51 2022] DELETE /identity/v3/projects/d7d00636950b44e7b6a4ef7496bae560 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:07:51.918412 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b63717-b85b-45d2-bee3-92364ac298db None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:51.918895 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b63717-b85b-45d2-bee3-92364ac298db None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:51.919349 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b63717-b85b-45d2-bee3-92364ac298db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:51.961526 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-53b63717-b85b-45d2-bee3-92364ac298db None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:51.963026 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-53b63717-b85b-45d2-bee3-92364ac298db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:52.042075 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-41b37289-f166-4094-8e98-97fc0f4477eb None None] MFA Rules not processed for user `469802150c6b4cb886eee2a33f618e2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:52.043329 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-41b37289-f166-4094-8e98-97fc0f4477eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:52.124516 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 748/1460] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:07:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:07:52.131973 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2455bc3c-2977-4f15-9fe4-2301df5f5937 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:52.132320 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2455bc3c-2977-4f15-9fe4-2301df5f5937 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:52.132611 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2455bc3c-2977-4f15-9fe4-2301df5f5937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:52.194696 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-2455bc3c-2977-4f15-9fe4-2301df5f5937 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:52.196043 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-2455bc3c-2977-4f15-9fe4-2301df5f5937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:52.208152 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 713/1461] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:07:51 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:07:52.209194 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 749/1462] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:07:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 03:07:52.211697 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b37289-f166-4094-8e98-97fc0f4477eb None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:52.212869 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2455bc3c-2977-4f15-9fe4-2301df5f5937 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:52.216272 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-41b37289-f166-4094-8e98-97fc0f4477eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:52.222771 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2455bc3c-2977-4f15-9fe4-2301df5f5937 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:52.229528 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc4d5e1-94c0-493f-929a-1cdbed5079be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:52.230490 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc4d5e1-94c0-493f-929a-1cdbed5079be admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:52.230935 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc4d5e1-94c0-493f-929a-1cdbed5079be admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:52.231346 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc4d5e1-94c0-493f-929a-1cdbed5079be admin admin] PATH_INFO: `/v3/domains` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:52.232133 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc4d5e1-94c0-493f-929a-1cdbed5079be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 03:07:52.232600 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc4d5e1-94c0-493f-929a-1cdbed5079be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:52.235456 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc4d5e1-94c0-493f-929a-1cdbed5079be admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:52.239568 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b37289-f166-4094-8e98-97fc0f4477eb None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:52.245717 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d389d090-ddd8-4370-8ea6-ab7937470c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:52.246590 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d389d090-ddd8-4370-8ea6-ab7937470c33 admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:52.246882 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d389d090-ddd8-4370-8ea6-ab7937470c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:52.247150 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d389d090-ddd8-4370-8ea6-ab7937470c33 admin admin] PATH_INFO: `/v3/projects` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:52.247792 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d389d090-ddd8-4370-8ea6-ab7937470c33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:52.251299 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d389d090-ddd8-4370-8ea6-ab7937470c33 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:52.280114 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 750/1463] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Oct 14 03:07:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 03:07:52.282215 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc4d5e1-94c0-493f-929a-1cdbed5079be admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:52.313933 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc4d5e1-94c0-493f-929a-1cdbed5079be admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:52.326876 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3316375-0ea0-420b-bd39-683340290f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:52.327703 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3316375-0ea0-420b-bd39-683340290f30 admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:52.328139 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3316375-0ea0-420b-bd39-683340290f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:52.328552 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3316375-0ea0-420b-bd39-683340290f30 admin admin] PATH_INFO: `/v3/projects` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:52.329332 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3316375-0ea0-420b-bd39-683340290f30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:52.331491 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3316375-0ea0-420b-bd39-683340290f30 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:52.420340 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 714/1464] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:07:52 2022] POST /identity/v3/projects => generated 365 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:07:52.421320 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d389d090-ddd8-4370-8ea6-ab7937470c33 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:52.487958 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 751/1465] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:07:52 2022] POST /identity/v3/projects => generated 369 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:07:52.492211 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3316375-0ea0-420b-bd39-683340290f30 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:52.576047 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d389d090-ddd8-4370-8ea6-ab7937470c33 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:52.663270 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3316375-0ea0-420b-bd39-683340290f30 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:52.680982 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b75d7a-565a-41ba-85a0-79944099e169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:52.681658 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b75d7a-565a-41ba-85a0-79944099e169 admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:52.681974 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b75d7a-565a-41ba-85a0-79944099e169 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:52.682254 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b75d7a-565a-41ba-85a0-79944099e169 admin admin] PATH_INFO: `/v3/users` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:52.682910 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b75d7a-565a-41ba-85a0-79944099e169 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:52.685238 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b75d7a-565a-41ba-85a0-79944099e169 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:52.727419 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65454fa8-861f-44c3-9a6c-8ac03ce4a60f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:52.727419 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65454fa8-861f-44c3-9a6c-8ac03ce4a60f admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:52.727419 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65454fa8-861f-44c3-9a6c-8ac03ce4a60f admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:52.727419 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65454fa8-861f-44c3-9a6c-8ac03ce4a60f admin admin] PATH_INFO: `/v3/users` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:52.730290 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65454fa8-861f-44c3-9a6c-8ac03ce4a60f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:52.730290 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65454fa8-861f-44c3-9a6c-8ac03ce4a60f admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:52.809720 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 752/1466] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:07:52 2022] POST /identity/v3/users => generated 285 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:07:52.813892 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 715/1467] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:07:52 2022] POST /identity/v3/users => generated 289 bytes in 393 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:07:52.820766 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b75d7a-565a-41ba-85a0-79944099e169 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:52.822842 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9246fffb-c1df-4a05-a13f-05a797c54052 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:52.823129 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9246fffb-c1df-4a05-a13f-05a797c54052 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:52.823406 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9246fffb-c1df-4a05-a13f-05a797c54052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:52.848549 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-69b75d7a-565a-41ba-85a0-79944099e169 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:52.889458 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-9246fffb-c1df-4a05-a13f-05a797c54052 None None] MFA Rules not processed for user `ab344e29f747423fb8766f2d8fbfef0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:52.895295 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:INSERT=3,keystone:DELETE=3,keystone:SELECT=94 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:07:52.895960 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-9246fffb-c1df-4a05-a13f-05a797c54052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:52.914233 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b75d7a-565a-41ba-85a0-79944099e169 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:52.936939 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56385d92-5de3-441c-9775-b082ef258c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:52.938228 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56385d92-5de3-441c-9775-b082ef258c9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:52.938634 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56385d92-5de3-441c-9775-b082ef258c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:52.939498 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56385d92-5de3-441c-9775-b082ef258c9e admin admin] PATH_INFO: `/v3/projects/7318cb48c37b44a9bcd1f857f936ecc1` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:52.943384 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56385d92-5de3-441c-9775-b082ef258c9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7318cb48c37b44a9bcd1f857f936ecc1)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:52.955098 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56385d92-5de3-441c-9775-b082ef258c9e admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:53.043763 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 716/1468] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:07:52 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:07:53.043763 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9246fffb-c1df-4a05-a13f-05a797c54052 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:53.075015 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9246fffb-c1df-4a05-a13f-05a797c54052 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:53.087528 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59842743-3415-4a2c-a90b-026ea9f3c5ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:53.088165 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59842743-3415-4a2c-a90b-026ea9f3c5ee admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:53.088447 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59842743-3415-4a2c-a90b-026ea9f3c5ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:53.088724 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59842743-3415-4a2c-a90b-026ea9f3c5ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:53.089385 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59842743-3415-4a2c-a90b-026ea9f3c5ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:53.091070 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59842743-3415-4a2c-a90b-026ea9f3c5ee admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:53.114258 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 717/1469] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:07:53 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:07:53.120659 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59842743-3415-4a2c-a90b-026ea9f3c5ee admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:53.139081 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59842743-3415-4a2c-a90b-026ea9f3c5ee admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:53.160924 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-56385d92-5de3-441c-9775-b082ef258c9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7318cb48c37b44a9bcd1f857f936ecc1', 'request_id': 'req-56385d92-5de3-441c-9775-b082ef258c9e'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:53.177266 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-56385d92-5de3-441c-9775-b082ef258c9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7318cb48c37b44a9bcd1f857f936ecc1', 'request_id': 'req-56385d92-5de3-441c-9775-b082ef258c9e'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:53.252956 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08e4ee5-498e-46d1-b66a-963b5032e7a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:53.253749 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08e4ee5-498e-46d1-b66a-963b5032e7a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:53.255414 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08e4ee5-498e-46d1-b66a-963b5032e7a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:53.258259 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08e4ee5-498e-46d1-b66a-963b5032e7a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:53.258259 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08e4ee5-498e-46d1-b66a-963b5032e7a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:53.260478 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08e4ee5-498e-46d1-b66a-963b5032e7a4 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:53.268813 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-56385d92-5de3-441c-9775-b082ef258c9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7318cb48c37b44a9bcd1f857f936ecc1', 'request_id': 'req-56385d92-5de3-441c-9775-b082ef258c9e'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:53.279842 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 718/1470] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:07:53 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:07:53.281203 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e7c02e-e412-4f0e-bc37-4d15223af86f None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:53.281492 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e7c02e-e412-4f0e-bc37-4d15223af86f None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:53.281860 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e7c02e-e412-4f0e-bc37-4d15223af86f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:53.300110 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-56385d92-5de3-441c-9775-b082ef258c9e admin admin] The token cache is being invalidate because project 7318cb48c37b44a9bcd1f857f936ecc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81517) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:07:53.300110 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-56385d92-5de3-441c-9775-b082ef258c9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:53.304152 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-56385d92-5de3-441c-9775-b082ef258c9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:07:53.309522 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 753/1471] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:07:52 2022] DELETE /identity/v3/projects/7318cb48c37b44a9bcd1f857f936ecc1 => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:07:53.310478 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56385d92-5de3-441c-9775-b082ef258c9e admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:53.311867 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-56385d92-5de3-441c-9775-b082ef258c9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:53.333463 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56385d92-5de3-441c-9775-b082ef258c9e admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:53.340498 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020fbe52-e8d5-4b81-88a3-4b59fdcc7da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:53.341419 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020fbe52-e8d5-4b81-88a3-4b59fdcc7da1 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:53.344013 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:07:53.344845 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020fbe52-e8d5-4b81-88a3-4b59fdcc7da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:53.345432 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020fbe52-e8d5-4b81-88a3-4b59fdcc7da1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:53.346543 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020fbe52-e8d5-4b81-88a3-4b59fdcc7da1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:53.349775 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020fbe52-e8d5-4b81-88a3-4b59fdcc7da1 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:53.387932 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-f6e7c02e-e412-4f0e-bc37-4d15223af86f None None] MFA Rules not processed for user `44b37050076b42e6af1a67bd7d6d06e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:53.391660 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-f6e7c02e-e412-4f0e-bc37-4d15223af86f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:53.395996 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 754/1472] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:07:53 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:07:53.396901 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020fbe52-e8d5-4b81-88a3-4b59fdcc7da1 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:53.404586 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-020fbe52-e8d5-4b81-88a3-4b59fdcc7da1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:53.438435 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020fbe52-e8d5-4b81-88a3-4b59fdcc7da1 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:53.450892 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083c04e9-2a1c-4392-a905-eef468fc8a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:53.451584 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c04e9-2a1c-4392-a905-eef468fc8a57 admin admin] REQUEST_METHOD: `PUT` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:53.451871 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c04e9-2a1c-4392-a905-eef468fc8a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:53.452204 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c04e9-2a1c-4392-a905-eef468fc8a57 admin admin] PATH_INFO: `/v3/projects/b808bf2e6e374292a020f2394a066af9/users/f8101d3da60542b9beeaff579a177c69/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:53.479385 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083c04e9-2a1c-4392-a905-eef468fc8a57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b808bf2e6e374292a020f2394a066af9, user_id=f8101d3da60542b9beeaff579a177c69, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:53.482186 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083c04e9-2a1c-4392-a905-eef468fc8a57 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:53.525448 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 719/1473] 10.222.0.39 () {58 vars in 1009 bytes} [Fri Oct 14 03:07:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:07:53.526934 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874b3e3e-980f-4da9-8709-7b1e99af5c72 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:53.529727 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874b3e3e-980f-4da9-8709-7b1e99af5c72 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:53.529727 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874b3e3e-980f-4da9-8709-7b1e99af5c72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:53.580357 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-874b3e3e-980f-4da9-8709-7b1e99af5c72 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:53.581622 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 755/1474] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:07:53 2022] PUT /identity/v3/projects/b808bf2e6e374292a020f2394a066af9/users/f8101d3da60542b9beeaff579a177c69/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:07:53.582236 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-874b3e3e-980f-4da9-8709-7b1e99af5c72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:53.582520 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083c04e9-2a1c-4392-a905-eef468fc8a57 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:53.753809 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 720/1475] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:07:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:07:53.753809 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874b3e3e-980f-4da9-8709-7b1e99af5c72 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:53.778409 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874b3e3e-980f-4da9-8709-7b1e99af5c72 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:53.778767 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083c04e9-2a1c-4392-a905-eef468fc8a57 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:53.786392 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84c07f7-0083-4b91-b973-6106afafe04e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:53.787478 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84c07f7-0083-4b91-b973-6106afafe04e admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:53.787919 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84c07f7-0083-4b91-b973-6106afafe04e admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:53.788337 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84c07f7-0083-4b91-b973-6106afafe04e admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:53.789662 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa401dbb-0e7e-4ef2-a6a2-ba1cd0c450d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:53.789982 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84c07f7-0083-4b91-b973-6106afafe04e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:53.790609 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa401dbb-0e7e-4ef2-a6a2-ba1cd0c450d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:53.790962 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa401dbb-0e7e-4ef2-a6a2-ba1cd0c450d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:53.791307 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa401dbb-0e7e-4ef2-a6a2-ba1cd0c450d5 admin admin] PATH_INFO: `/v3/projects/1480461d859a423fb589e6baa6438d81/users/3c19194a8b4240b483ce02934843dc76/roles/9bd0470b544240209a420f640c79c06c` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:53.793098 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84c07f7-0083-4b91-b973-6106afafe04e admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:53.813345 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa401dbb-0e7e-4ef2-a6a2-ba1cd0c450d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1480461d859a423fb589e6baa6438d81, user_id=3c19194a8b4240b483ce02934843dc76, role_id=9bd0470b544240209a420f640c79c06c)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:53.816534 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa401dbb-0e7e-4ef2-a6a2-ba1cd0c450d5 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:53.821209 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 721/1476] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:07:53 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:07:53.824391 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a37a139-6c4d-43f7-97fd-26d05c469ff7 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:53.826226 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a37a139-6c4d-43f7-97fd-26d05c469ff7 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:53.826720 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a37a139-6c4d-43f7-97fd-26d05c469ff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:53.853979 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-4a37a139-6c4d-43f7-97fd-26d05c469ff7 None None] MFA Rules not processed for user `fb1c279c2f7f404ab9c846a3ba594886`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:53.855327 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-4a37a139-6c4d-43f7-97fd-26d05c469ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:53.886863 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 756/1477] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:07:53 2022] PUT /identity/v3/projects/1480461d859a423fb589e6baa6438d81/users/3c19194a8b4240b483ce02934843dc76/roles/9bd0470b544240209a420f640c79c06c => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:07:53.888516 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de538052-fd3d-46ef-a824-769efff82721 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:53.889086 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de538052-fd3d-46ef-a824-769efff82721 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:53.889631 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de538052-fd3d-46ef-a824-769efff82721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:53.925775 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-de538052-fd3d-46ef-a824-769efff82721 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:53.929232 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-de538052-fd3d-46ef-a824-769efff82721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:53.963020 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 722/1478] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:07:53 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:07:53.963911 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a37a139-6c4d-43f7-97fd-26d05c469ff7 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:53.986037 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a37a139-6c4d-43f7-97fd-26d05c469ff7 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:53.989819 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:07:54.025903 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 757/1479] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:07:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:07:54.028076 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de538052-fd3d-46ef-a824-769efff82721 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:54.030713 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-de538052-fd3d-46ef-a824-769efff82721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:54.110688 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487707f9-a45e-4938-bc92-092dd8ad9140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:54.111465 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487707f9-a45e-4938-bc92-092dd8ad9140 admin admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:54.111750 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487707f9-a45e-4938-bc92-092dd8ad9140 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:54.112029 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487707f9-a45e-4938-bc92-092dd8ad9140 admin admin] PATH_INFO: `/v3/projects/b808bf2e6e374292a020f2394a066af9/users/f8101d3da60542b9beeaff579a177c69/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:54.117968 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487707f9-a45e-4938-bc92-092dd8ad9140 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b808bf2e6e374292a020f2394a066af9, user_id=f8101d3da60542b9beeaff579a177c69, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:54.121499 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487707f9-a45e-4938-bc92-092dd8ad9140 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:54.137242 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 723/1480] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:07:53 2022] PUT /identity/v3/projects/b808bf2e6e374292a020f2394a066af9/users/f8101d3da60542b9beeaff579a177c69/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:07:54.138838 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3b61bc-35f1-4aa9-bb98-0d2f3be30877 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:54.139149 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3b61bc-35f1-4aa9-bb98-0d2f3be30877 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:54.139401 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3b61bc-35f1-4aa9-bb98-0d2f3be30877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:54.195751 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-be3b61bc-35f1-4aa9-bb98-0d2f3be30877 None None] MFA Rules not processed for user `3c19194a8b4240b483ce02934843dc76`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:54.197516 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:07:54.201911 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-be3b61bc-35f1-4aa9-bb98-0d2f3be30877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:54.217297 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de538052-fd3d-46ef-a824-769efff82721 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:54.338319 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 724/1481] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:07:54 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:07:54.343541 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3b61bc-35f1-4aa9-bb98-0d2f3be30877 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:54.353595 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1494bc7f-4319-4f8b-8b3d-bce12e2d95c2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5e07da64804243bc18862072b027a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:54.354268 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1494bc7f-4319-4f8b-8b3d-bce12e2d95c2 service swift] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:54.354638 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1494bc7f-4319-4f8b-8b3d-bce12e2d95c2 service swift] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:54.354819 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1494bc7f-4319-4f8b-8b3d-bce12e2d95c2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:54.474191 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3b61bc-35f1-4aa9-bb98-0d2f3be30877 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:54.475504 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1494bc7f-4319-4f8b-8b3d-bce12e2d95c2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:54.479458 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1494bc7f-4319-4f8b-8b3d-bce12e2d95c2 service swift] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:54.485480 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6cd78d-8060-4577-9fa3-aeb31d4a0900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:54.486294 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6cd78d-8060-4577-9fa3-aeb31d4a0900 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:54.486575 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6cd78d-8060-4577-9fa3-aeb31d4a0900 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:54.487980 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6cd78d-8060-4577-9fa3-aeb31d4a0900 admin admin] PATH_INFO: `/v3/domains` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:54.488679 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6cd78d-8060-4577-9fa3-aeb31d4a0900 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 03:07:54.489071 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6cd78d-8060-4577-9fa3-aeb31d4a0900 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:54.492656 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6cd78d-8060-4577-9fa3-aeb31d4a0900 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:54.500266 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 758/1482] 10.222.0.39 () {60 vars in 1384 bytes} [Fri Oct 14 03:07:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 14 03:07:54.506069 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea78a428-4652-4e1a-8731-84b9b2cee7d0 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:54.506069 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea78a428-4652-4e1a-8731-84b9b2cee7d0 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:54.506069 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea78a428-4652-4e1a-8731-84b9b2cee7d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:54.522005 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 725/1483] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Oct 14 03:07:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 03:07:54.528254 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6cd78d-8060-4577-9fa3-aeb31d4a0900 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:54.556756 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6cd78d-8060-4577-9fa3-aeb31d4a0900 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:54.568812 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0295e127-2283-46fc-ba1c-9a16b97ddc63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:54.569440 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0295e127-2283-46fc-ba1c-9a16b97ddc63 admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:54.569922 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0295e127-2283-46fc-ba1c-9a16b97ddc63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:54.571770 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0295e127-2283-46fc-ba1c-9a16b97ddc63 admin admin] PATH_INFO: `/v3/projects` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:54.573614 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0295e127-2283-46fc-ba1c-9a16b97ddc63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:54.578008 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-ea78a428-4652-4e1a-8731-84b9b2cee7d0 None None] MFA Rules not processed for user `f8101d3da60542b9beeaff579a177c69`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:54.579043 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0295e127-2283-46fc-ba1c-9a16b97ddc63 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:54.580774 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-ea78a428-4652-4e1a-8731-84b9b2cee7d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:54.629328 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:07:54.745624 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 726/1484] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:07:54 2022] POST /identity/v3/projects => generated 367 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:07:54.746507 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0295e127-2283-46fc-ba1c-9a16b97ddc63 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:54.835313 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 759/1485] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:07:54 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:07:54.835904 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea78a428-4652-4e1a-8731-84b9b2cee7d0 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:54.885461 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea78a428-4652-4e1a-8731-84b9b2cee7d0 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:54.939742 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0295e127-2283-46fc-ba1c-9a16b97ddc63 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:54.958185 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c34670-1f55-4651-a49f-b6ccae97690a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:54.960108 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c34670-1f55-4651-a49f-b6ccae97690a admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:54.960600 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c34670-1f55-4651-a49f-b6ccae97690a admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:54.962808 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c34670-1f55-4651-a49f-b6ccae97690a admin admin] PATH_INFO: `/v3/projects` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:54.963874 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c34670-1f55-4651-a49f-b6ccae97690a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:54.971628 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c34670-1f55-4651-a49f-b6ccae97690a admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:55.105706 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:07:55.140643 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 727/1486] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:07:54 2022] POST /identity/v3/projects => generated 389 bytes in 394 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:07:55.142328 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c34670-1f55-4651-a49f-b6ccae97690a admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:55.144093 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-32c34670-1f55-4651-a49f-b6ccae97690a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:55.318216 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c34670-1f55-4651-a49f-b6ccae97690a admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:55.338705 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fb6558-b01f-4379-8cf8-2f689850f792 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab599d98b0624a2e89090312bcab713f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:55.341952 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fb6558-b01f-4379-8cf8-2f689850f792 service cinder] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:55.344570 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fb6558-b01f-4379-8cf8-2f689850f792 service cinder] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:55.346814 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fb6558-b01f-4379-8cf8-2f689850f792 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:55.410984 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:07:55.499458 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fd19f8-fd76-4658-8cb5-119b26ecd827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:55.500255 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fd19f8-fd76-4658-8cb5-119b26ecd827 admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:55.500648 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fd19f8-fd76-4658-8cb5-119b26ecd827 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:55.501063 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fd19f8-fd76-4658-8cb5-119b26ecd827 admin admin] PATH_INFO: `/v3/users` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:55.502103 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fd19f8-fd76-4658-8cb5-119b26ecd827 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:55.505050 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fd19f8-fd76-4658-8cb5-119b26ecd827 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:55.551124 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fb6558-b01f-4379-8cf8-2f689850f792 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:55.552516 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fb6558-b01f-4379-8cf8-2f689850f792 service cinder] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:55.576016 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 728/1487] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:07:55 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:07:55.576893 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fb6558-b01f-4379-8cf8-2f689850f792 service cinder] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:55.613552 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 760/1488] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:07:54 2022] POST /identity/v3/users => generated 290 bytes in 775 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:07:55.615090 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fd19f8-fd76-4658-8cb5-119b26ecd827 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:55.655058 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fd19f8-fd76-4658-8cb5-119b26ecd827 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:55.724374 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fb6558-b01f-4379-8cf8-2f689850f792 service cinder] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:55.728992 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614e68eb-5889-45c2-8026-0898462860f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:55.729515 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614e68eb-5889-45c2-8026-0898462860f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:55.729828 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614e68eb-5889-45c2-8026-0898462860f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:55.730227 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614e68eb-5889-45c2-8026-0898462860f9 admin admin] PATH_INFO: `/v3/users` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:55.730757 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614e68eb-5889-45c2-8026-0898462860f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:55.732731 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614e68eb-5889-45c2-8026-0898462860f9 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:55.739575 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7763feb9-1691-46ad-91e8-8a1a519bb6f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:55.740318 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7763feb9-1691-46ad-91e8-8a1a519bb6f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:55.740783 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7763feb9-1691-46ad-91e8-8a1a519bb6f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:55.741130 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7763feb9-1691-46ad-91e8-8a1a519bb6f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:55.743024 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7763feb9-1691-46ad-91e8-8a1a519bb6f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:55.744030 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7763feb9-1691-46ad-91e8-8a1a519bb6f7 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:55.777668 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 761/1489] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:07:55 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:07:55.783410 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7763feb9-1691-46ad-91e8-8a1a519bb6f7 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:55.812114 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7763feb9-1691-46ad-91e8-8a1a519bb6f7 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:55.840990 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 729/1490] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:07:55 2022] POST /identity/v3/users => generated 295 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:07:55.845206 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671a9459-aa59-435f-8917-a90145866ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:55.845363 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614e68eb-5889-45c2-8026-0898462860f9 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:55.846605 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a9459-aa59-435f-8917-a90145866ce0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:55.848347 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a9459-aa59-435f-8917-a90145866ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:55.848868 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a9459-aa59-435f-8917-a90145866ce0 admin admin] PATH_INFO: `/v3/projects/e68fd16aec1f444293ba24772452ed5b/users/eb00b9de1cce48a093b7de2bfe2adbd0/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:55.880617 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614e68eb-5889-45c2-8026-0898462860f9 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:55.896357 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671a9459-aa59-435f-8917-a90145866ce0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e68fd16aec1f444293ba24772452ed5b, user_id=eb00b9de1cce48a093b7de2bfe2adbd0, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:55.903190 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f30d09-919b-49e3-ae0f-47c25a765b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:55.903847 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f30d09-919b-49e3-ae0f-47c25a765b55 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:55.904137 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f30d09-919b-49e3-ae0f-47c25a765b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:55.904476 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f30d09-919b-49e3-ae0f-47c25a765b55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:55.905171 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f30d09-919b-49e3-ae0f-47c25a765b55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:55.907976 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671a9459-aa59-435f-8917-a90145866ce0 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:55.921162 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f30d09-919b-49e3-ae0f-47c25a765b55 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:55.925947 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:07:55.961924 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 730/1491] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:07:55 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:07:55.963398 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f30d09-919b-49e3-ae0f-47c25a765b55 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:55.980792 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f30d09-919b-49e3-ae0f-47c25a765b55 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:55.990147 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de96d8df-be5a-4bfc-b52f-c06b072f6071 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab599d98b0624a2e89090312bcab713f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:55.990782 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de96d8df-be5a-4bfc-b52f-c06b072f6071 service cinder] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:55.991130 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de96d8df-be5a-4bfc-b52f-c06b072f6071 service cinder] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:55.991414 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de96d8df-be5a-4bfc-b52f-c06b072f6071 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:55.995929 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-de96d8df-be5a-4bfc-b52f-c06b072f6071 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:56.053075 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 762/1492] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:07:55 2022] PUT /identity/v3/projects/e68fd16aec1f444293ba24772452ed5b/users/eb00b9de1cce48a093b7de2bfe2adbd0/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:07:56.055414 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671a9459-aa59-435f-8917-a90145866ce0 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:56.197546 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de96d8df-be5a-4bfc-b52f-c06b072f6071 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:56.197546 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de96d8df-be5a-4bfc-b52f-c06b072f6071 service cinder] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:56.197546 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 731/1493] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:07:55 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:07:56.197546 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de96d8df-be5a-4bfc-b52f-c06b072f6071 service cinder] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:56.218254 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de96d8df-be5a-4bfc-b52f-c06b072f6071 service cinder] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:56.311768 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671a9459-aa59-435f-8917-a90145866ce0 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:56.327701 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3808cb7d-05cd-42c2-8782-ffa3e69698d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:56.328654 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3808cb7d-05cd-42c2-8782-ffa3e69698d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:56.329159 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3808cb7d-05cd-42c2-8782-ffa3e69698d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:56.329639 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3808cb7d-05cd-42c2-8782-ffa3e69698d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:56.330541 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3808cb7d-05cd-42c2-8782-ffa3e69698d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:56.332344 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3808cb7d-05cd-42c2-8782-ffa3e69698d4 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:56.358175 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 763/1494] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:07:56 2022] GET /identity/v3/roles => generated 1420 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:07:56.358616 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51224693-a443-420b-889c-c293a2748afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:56.359570 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51224693-a443-420b-889c-c293a2748afa admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:56.360019 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51224693-a443-420b-889c-c293a2748afa admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:56.361060 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3808cb7d-05cd-42c2-8782-ffa3e69698d4 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:56.362665 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51224693-a443-420b-889c-c293a2748afa admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:56.363417 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51224693-a443-420b-889c-c293a2748afa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:56.367090 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51224693-a443-420b-889c-c293a2748afa admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:56.393718 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3808cb7d-05cd-42c2-8782-ffa3e69698d4 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:56.403236 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 732/1495] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:07:56 2022] GET /identity/v3/roles => generated 1420 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:07:56.405574 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51224693-a443-420b-889c-c293a2748afa admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:56.408918 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaf063c-6b84-40b2-bfd6-29bb1bf2a792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:56.410587 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaf063c-6b84-40b2-bfd6-29bb1bf2a792 admin admin] REQUEST_METHOD: `PUT` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:56.411046 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaf063c-6b84-40b2-bfd6-29bb1bf2a792 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:56.411998 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaf063c-6b84-40b2-bfd6-29bb1bf2a792 admin admin] PATH_INFO: `/v3/projects/bf74e7fe6f2d49d4adbae394b68be44b/users/7f80dd0aaf2642db89dabbc4951a917f/roles/9bd0470b544240209a420f640c79c06c` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:56.428027 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51224693-a443-420b-889c-c293a2748afa admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:56.435177 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc2fc75-f6bd-4547-a526-db0e59ba2916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:56.435986 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc2fc75-f6bd-4547-a526-db0e59ba2916 admin admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:56.436277 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc2fc75-f6bd-4547-a526-db0e59ba2916 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:56.436554 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc2fc75-f6bd-4547-a526-db0e59ba2916 admin admin] PATH_INFO: `/v3/projects/e68fd16aec1f444293ba24772452ed5b/users/eb00b9de1cce48a093b7de2bfe2adbd0/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:56.439227 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc2fc75-f6bd-4547-a526-db0e59ba2916 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e68fd16aec1f444293ba24772452ed5b, user_id=eb00b9de1cce48a093b7de2bfe2adbd0, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:56.442721 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc2fc75-f6bd-4547-a526-db0e59ba2916 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:56.468934 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 733/1496] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:07:56 2022] PUT /identity/v3/projects/e68fd16aec1f444293ba24772452ed5b/users/eb00b9de1cce48a093b7de2bfe2adbd0/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:07:56.472581 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d6e8f6-c4c2-4982-bc61-c310336374f8 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:56.472877 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d6e8f6-c4c2-4982-bc61-c310336374f8 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:56.473161 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d6e8f6-c4c2-4982-bc61-c310336374f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:56.473352 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaf063c-6b84-40b2-bfd6-29bb1bf2a792 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf74e7fe6f2d49d4adbae394b68be44b, user_id=7f80dd0aaf2642db89dabbc4951a917f, role_id=9bd0470b544240209a420f640c79c06c)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:56.480480 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaf063c-6b84-40b2-bfd6-29bb1bf2a792 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:56.508331 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-d5d6e8f6-c4c2-4982-bc61-c310336374f8 None None] MFA Rules not processed for user `eb00b9de1cce48a093b7de2bfe2adbd0`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:56.511112 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-d5d6e8f6-c4c2-4982-bc61-c310336374f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:56.680021 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 764/1497] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:07:56 2022] PUT /identity/v3/projects/bf74e7fe6f2d49d4adbae394b68be44b/users/7f80dd0aaf2642db89dabbc4951a917f/roles/9bd0470b544240209a420f640c79c06c => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:07:56.689990 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc48b96-4cd6-4c70-ae44-215dede06f56 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:56.690508 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc48b96-4cd6-4c70-ae44-215dede06f56 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:56.690963 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc48b96-4cd6-4c70-ae44-215dede06f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:56.731018 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-4cc48b96-4cd6-4c70-ae44-215dede06f56 None None] MFA Rules not processed for user `7f80dd0aaf2642db89dabbc4951a917f`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:56.732794 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-4cc48b96-4cd6-4c70-ae44-215dede06f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:56.767034 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 734/1498] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:07:56 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:07:56.768606 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615a37f4-77d9-4f90-9a07-b3b6afcee84d None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:56.770387 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:07:56.774490 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615a37f4-77d9-4f90-9a07-b3b6afcee84d None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:56.778291 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615a37f4-77d9-4f90-9a07-b3b6afcee84d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:56.824904 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-615a37f4-77d9-4f90-9a07-b3b6afcee84d None None] MFA Rules not processed for user `b123c412ac714b06929a4bd86eca71e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:56.826585 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-615a37f4-77d9-4f90-9a07-b3b6afcee84d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:56.914671 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 765/1499] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:07:56 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 03:07:56.916037 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc48b96-4cd6-4c70-ae44-215dede06f56 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:56.930785 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-4cc48b96-4cd6-4c70-ae44-215dede06f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:56.938786 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 735/1500] 10.222.0.40 () {58 vars in 1015 bytes} [Fri Oct 14 03:07:56 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:07:56.945208 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615a37f4-77d9-4f90-9a07-b3b6afcee84d None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:57.110284 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc48b96-4cd6-4c70-ae44-215dede06f56 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:57.129591 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e50c1fc-8b7f-456a-9fc5-4e95cd4e852a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab599d98b0624a2e89090312bcab713f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:57.130472 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e50c1fc-8b7f-456a-9fc5-4e95cd4e852a service cinder] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:57.130810 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e50c1fc-8b7f-456a-9fc5-4e95cd4e852a service cinder] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:57.131144 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e50c1fc-8b7f-456a-9fc5-4e95cd4e852a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:57.192144 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615a37f4-77d9-4f90-9a07-b3b6afcee84d None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:57.199243 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4d8ef3-124b-444d-b8c2-a93bd010dae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:57.199969 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4d8ef3-124b-444d-b8c2-a93bd010dae7 admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:57.200321 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4d8ef3-124b-444d-b8c2-a93bd010dae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:57.200699 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4d8ef3-124b-444d-b8c2-a93bd010dae7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:57.201587 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4d8ef3-124b-444d-b8c2-a93bd010dae7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:57.203968 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4d8ef3-124b-444d-b8c2-a93bd010dae7 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:57.356443 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e50c1fc-8b7f-456a-9fc5-4e95cd4e852a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:57.363211 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e50c1fc-8b7f-456a-9fc5-4e95cd4e852a service cinder] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:57.395645 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 736/1501] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:07:56 2022] POST /identity/v3/projects => generated 393 bytes in 456 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:07:57.399326 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4d8ef3-124b-444d-b8c2-a93bd010dae7 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:57.401043 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-6d4d8ef3-124b-444d-b8c2-a93bd010dae7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:57.418268 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:07:57.419192 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 766/1502] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:07:56 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:07:57.420316 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e50c1fc-8b7f-456a-9fc5-4e95cd4e852a service cinder] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:57.448263 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:07:57.500517 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e50c1fc-8b7f-456a-9fc5-4e95cd4e852a service cinder] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:57.560431 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4d8ef3-124b-444d-b8c2-a93bd010dae7 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:57.578311 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89d46fe-53bc-45d6-a855-9372d35a5cbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:57.578955 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d46fe-53bc-45d6-a855-9372d35a5cbd service neutron] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:57.579338 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d46fe-53bc-45d6-a855-9372d35a5cbd service neutron] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:57.579513 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d46fe-53bc-45d6-a855-9372d35a5cbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:57.597370 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89d46fe-53bc-45d6-a855-9372d35a5cbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:57.597564 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27382550-0c68-48c7-aed7-cc3cbf270513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:57.598427 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27382550-0c68-48c7-aed7-cc3cbf270513 admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:57.598738 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27382550-0c68-48c7-aed7-cc3cbf270513 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:57.598845 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89d46fe-53bc-45d6-a855-9372d35a5cbd service neutron] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:57.599022 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27382550-0c68-48c7-aed7-cc3cbf270513 admin admin] PATH_INFO: `/v3/users` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:57.599733 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27382550-0c68-48c7-aed7-cc3cbf270513 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:57.602523 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27382550-0c68-48c7-aed7-cc3cbf270513 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:57.619540 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 737/1503] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:07:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:07:57.704883 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89d46fe-53bc-45d6-a855-9372d35a5cbd service neutron] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:57.705290 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 767/1504] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:07:57 2022] POST /identity/v3/users => generated 303 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:07:57.709051 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27382550-0c68-48c7-aed7-cc3cbf270513 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:57.742038 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27382550-0c68-48c7-aed7-cc3cbf270513 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:57.759867 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d20d176-2c24-4afc-9857-d340149e3481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:57.759867 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d20d176-2c24-4afc-9857-d340149e3481 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:57.759867 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d20d176-2c24-4afc-9857-d340149e3481 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:57.759867 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d20d176-2c24-4afc-9857-d340149e3481 admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:57.760343 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d20d176-2c24-4afc-9857-d340149e3481 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:57.765166 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d20d176-2c24-4afc-9857-d340149e3481 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:57.807901 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 768/1505] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:07:57 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 03:07:57.813794 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d20d176-2c24-4afc-9857-d340149e3481 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:57.837672 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d20d176-2c24-4afc-9857-d340149e3481 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:57.859222 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b8045e-a16d-4bf3-9fb8-97ae7e471ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:57.859849 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b8045e-a16d-4bf3-9fb8-97ae7e471ccc admin admin] REQUEST_METHOD: `PUT` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:57.860147 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b8045e-a16d-4bf3-9fb8-97ae7e471ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:57.860862 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b8045e-a16d-4bf3-9fb8-97ae7e471ccc admin admin] PATH_INFO: `/v3/projects/38b83be5aa60429fb8a69bd5d2f6b8dc/users/df5d9a3a1d084be8bda8eebb1c53bf00/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:57.914267 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b8045e-a16d-4bf3-9fb8-97ae7e471ccc admin admin] RBAC: Authorizing `identity:create_grant(project_id=38b83be5aa60429fb8a69bd5d2f6b8dc, user_id=df5d9a3a1d084be8bda8eebb1c53bf00, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:57.923129 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b8045e-a16d-4bf3-9fb8-97ae7e471ccc admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:57.944997 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89d46fe-53bc-45d6-a855-9372d35a5cbd service neutron] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:57.960906 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670a6722-a0ab-4a29-829d-0e2a1827108f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab599d98b0624a2e89090312bcab713f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:57.964144 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670a6722-a0ab-4a29-829d-0e2a1827108f service cinder] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:57.964945 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670a6722-a0ab-4a29-829d-0e2a1827108f service cinder] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:57.965231 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670a6722-a0ab-4a29-829d-0e2a1827108f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:58.029072 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 769/1506] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:07:57 2022] PUT /identity/v3/projects/38b83be5aa60429fb8a69bd5d2f6b8dc/users/df5d9a3a1d084be8bda8eebb1c53bf00/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:07:58.030171 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b8045e-a16d-4bf3-9fb8-97ae7e471ccc admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:58.233214 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670a6722-a0ab-4a29-829d-0e2a1827108f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:58.235983 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670a6722-a0ab-4a29-829d-0e2a1827108f service cinder] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:58.293556 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 738/1507] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:07:57 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 589 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 03:07:58.332568 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b8045e-a16d-4bf3-9fb8-97ae7e471ccc admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:58.364703 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801993ff-8b87-4293-980d-d046d3ef21de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:58.366061 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801993ff-8b87-4293-980d-d046d3ef21de admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:58.366500 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801993ff-8b87-4293-980d-d046d3ef21de admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:58.370315 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801993ff-8b87-4293-980d-d046d3ef21de admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:58.373151 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801993ff-8b87-4293-980d-d046d3ef21de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:58.378606 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801993ff-8b87-4293-980d-d046d3ef21de admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:58.422176 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670a6722-a0ab-4a29-829d-0e2a1827108f service cinder] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:07:58.422883 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:07:58.444256 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670a6722-a0ab-4a29-829d-0e2a1827108f service cinder] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:07:58.444924 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 770/1508] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:07:58 2022] GET /identity/v3/roles => generated 1420 bytes in 413 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:07:58.451693 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:07:58.468897 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2649b9-d292-48d5-a5d7-c0f71435ca11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:07:58.470358 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2649b9-d292-48d5-a5d7-c0f71435ca11 admin admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:58.470647 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2649b9-d292-48d5-a5d7-c0f71435ca11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:58.475359 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2649b9-d292-48d5-a5d7-c0f71435ca11 admin admin] PATH_INFO: `/v3/projects/38b83be5aa60429fb8a69bd5d2f6b8dc/users/df5d9a3a1d084be8bda8eebb1c53bf00/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:58.496459 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2649b9-d292-48d5-a5d7-c0f71435ca11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38b83be5aa60429fb8a69bd5d2f6b8dc, user_id=df5d9a3a1d084be8bda8eebb1c53bf00, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:07:58.500350 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2649b9-d292-48d5-a5d7-c0f71435ca11 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:07:58.558621 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5fae8b-6ddd-48b7-82a2-fa2e7b162cf5 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:07:58.558920 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5fae8b-6ddd-48b7-82a2-fa2e7b162cf5 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:07:58.559193 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5fae8b-6ddd-48b7-82a2-fa2e7b162cf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:07:58.565158 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 739/1509] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:07:58 2022] PUT /identity/v3/projects/38b83be5aa60429fb8a69bd5d2f6b8dc/users/df5d9a3a1d084be8bda8eebb1c53bf00/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:07:58.638696 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-5b5fae8b-6ddd-48b7-82a2-fa2e7b162cf5 None None] MFA Rules not processed for user `df5d9a3a1d084be8bda8eebb1c53bf00`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:07:58.642938 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-5b5fae8b-6ddd-48b7-82a2-fa2e7b162cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:07:58.902616 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 771/1510] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:07:58 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:07:59.804559 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2649b9-d292-48d5-a5d7-c0f71435ca11 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:00.001389 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:00.013292 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2649b9-d292-48d5-a5d7-c0f71435ca11 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:00.018343 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed92e1b-0ed5-4583-b770-9a0a969fee09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab599d98b0624a2e89090312bcab713f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:00.019244 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed92e1b-0ed5-4583-b770-9a0a969fee09 service cinder] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:00.019686 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed92e1b-0ed5-4583-b770-9a0a969fee09 service cinder] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:00.020056 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed92e1b-0ed5-4583-b770-9a0a969fee09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:00.037559 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed92e1b-0ed5-4583-b770-9a0a969fee09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:00.039336 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed92e1b-0ed5-4583-b770-9a0a969fee09 service cinder] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:00.055375 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 740/1511] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:07:59 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:08:00.183902 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5fae8b-6ddd-48b7-82a2-fa2e7b162cf5 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:00.212470 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5fae8b-6ddd-48b7-82a2-fa2e7b162cf5 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:00.231182 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1285d3-0560-43da-a1b3-f62772c2ff31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab599d98b0624a2e89090312bcab713f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:00.231821 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1285d3-0560-43da-a1b3-f62772c2ff31 service cinder] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:00.232106 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1285d3-0560-43da-a1b3-f62772c2ff31 service cinder] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:00.232383 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1285d3-0560-43da-a1b3-f62772c2ff31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:00.320453 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:00.472978 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1285d3-0560-43da-a1b3-f62772c2ff31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:00.481658 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1285d3-0560-43da-a1b3-f62772c2ff31 service cinder] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:00.517381 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 772/1512] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:08:00 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:08:00.666919 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e6663c-5896-47ac-aa96-d135507e5073 None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:00.667998 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e6663c-5896-47ac-aa96-d135507e5073 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:00.668622 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e6663c-5896-47ac-aa96-d135507e5073 None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:00.670680 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 741/1513] 10.222.0.39 () {54 vars in 921 bytes} [Fri Oct 14 03:08:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 03:08:00.871645 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fc58f1-aaa5-44f1-84c0-d3e354563f02 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:00.873115 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fc58f1-aaa5-44f1-84c0-d3e354563f02 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:00.873115 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fc58f1-aaa5-44f1-84c0-d3e354563f02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:00.963984 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-25fc58f1-aaa5-44f1-84c0-d3e354563f02 None None] MFA Rules not processed for user `a6b07bf0faca47b28b50990d1720853f`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:08:00.970192 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-25fc58f1-aaa5-44f1-84c0-d3e354563f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:01.135023 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 773/1514] 10.222.0.39 () {58 vars in 1018 bytes} [Fri Oct 14 03:08:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:08:01.311694 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e6663c-5896-47ac-aa96-d135507e5073 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:01.317672 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-03e6663c-5896-47ac-aa96-d135507e5073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:01.385925 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e6663c-5896-47ac-aa96-d135507e5073 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:01.406750 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a50180-60a2-4749-80b4-87fe3bf2e52d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b07bf0faca47b28b50990d1720853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:01.407492 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a50180-60a2-4749-80b4-87fe3bf2e52d service nova] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:01.408173 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a50180-60a2-4749-80b4-87fe3bf2e52d service nova] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:01.408657 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a50180-60a2-4749-80b4-87fe3bf2e52d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:01.428802 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a50180-60a2-4749-80b4-87fe3bf2e52d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:01.432969 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a50180-60a2-4749-80b4-87fe3bf2e52d service nova] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:01.463675 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 742/1515] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:08:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:08:03.663024 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fb19c2-873f-4f74-8fab-2d982fbc38ed None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:03.663947 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fb19c2-873f-4f74-8fab-2d982fbc38ed None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:03.664465 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fb19c2-873f-4f74-8fab-2d982fbc38ed None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:03.666904 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 774/1516] 10.222.0.39 () {54 vars in 921 bytes} [Fri Oct 14 03:08:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 03:08:03.737097 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5247a10-08cc-43aa-b431-e0de521aba3e None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:03.737606 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5247a10-08cc-43aa-b431-e0de521aba3e None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:03.738119 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5247a10-08cc-43aa-b431-e0de521aba3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:03.787072 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-d5247a10-08cc-43aa-b431-e0de521aba3e None None] MFA Rules not processed for user `a6b07bf0faca47b28b50990d1720853f`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:08:03.793244 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-d5247a10-08cc-43aa-b431-e0de521aba3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:03.832915 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 743/1517] 10.222.0.39 () {58 vars in 1018 bytes} [Fri Oct 14 03:08:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:08:04.313940 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fb19c2-873f-4f74-8fab-2d982fbc38ed None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:04.315482 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-c4fb19c2-873f-4f74-8fab-2d982fbc38ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:04.363033 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fb19c2-873f-4f74-8fab-2d982fbc38ed None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:04.375137 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ca0ba7-fce2-4c2f-ab06-53cc28a2a816 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b07bf0faca47b28b50990d1720853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:04.375825 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ca0ba7-fce2-4c2f-ab06-53cc28a2a816 service nova] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:04.376105 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ca0ba7-fce2-4c2f-ab06-53cc28a2a816 service nova] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:04.376381 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ca0ba7-fce2-4c2f-ab06-53cc28a2a816 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:04.398029 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ca0ba7-fce2-4c2f-ab06-53cc28a2a816 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:04.400280 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ca0ba7-fce2-4c2f-ab06-53cc28a2a816 service nova] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:04.441821 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 775/1518] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:08:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:08:05.957885 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5247a10-08cc-43aa-b431-e0de521aba3e None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:06.051862 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5247a10-08cc-43aa-b431-e0de521aba3e None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:06.057327 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a570a1-243f-4f38-b5e9-48baafb5c160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:06.058156 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a570a1-243f-4f38-b5e9-48baafb5c160 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:06.058510 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a570a1-243f-4f38-b5e9-48baafb5c160 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:06.058906 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a570a1-243f-4f38-b5e9-48baafb5c160 admin admin] PATH_INFO: `/v3/users/3c19194a8b4240b483ce02934843dc76` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:06.060099 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a570a1-243f-4f38-b5e9-48baafb5c160 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c19194a8b4240b483ce02934843dc76)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:06.062279 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a570a1-243f-4f38-b5e9-48baafb5c160 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:06.242328 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-61a570a1-243f-4f38-b5e9-48baafb5c160 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c19194a8b4240b483ce02934843dc76'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:06.249785 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-61a570a1-243f-4f38-b5e9-48baafb5c160 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c19194a8b4240b483ce02934843dc76'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:06.263536 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-61a570a1-243f-4f38-b5e9-48baafb5c160 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c19194a8b4240b483ce02934843dc76'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:06.264269 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-61a570a1-243f-4f38-b5e9-48baafb5c160 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c19194a8b4240b483ce02934843dc76'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:06.324497 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-61a570a1-243f-4f38-b5e9-48baafb5c160 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c19194a8b4240b483ce02934843dc76'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:06.330753 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 744/1519] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:08:05 2022] DELETE /identity/v3/users/3c19194a8b4240b483ce02934843dc76 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:08:06.335079 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f841e8ea-6912-4a51-b14a-14ac92c440f7 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:06.335551 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f841e8ea-6912-4a51-b14a-14ac92c440f7 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:06.335647 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f841e8ea-6912-4a51-b14a-14ac92c440f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:06.355045 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-f841e8ea-6912-4a51-b14a-14ac92c440f7 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:08:06.356239 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-f841e8ea-6912-4a51-b14a-14ac92c440f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:06.414874 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 776/1520] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:08:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 03:08:06.424609 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a570a1-243f-4f38-b5e9-48baafb5c160 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:06.426039 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-61a570a1-243f-4f38-b5e9-48baafb5c160 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:06.483819 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a570a1-243f-4f38-b5e9-48baafb5c160 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:06.487812 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0326aa-106a-4c3a-b738-9339ea3358b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:06.488418 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0326aa-106a-4c3a-b738-9339ea3358b9 service neutron] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:06.488701 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0326aa-106a-4c3a-b738-9339ea3358b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:06.488981 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0326aa-106a-4c3a-b738-9339ea3358b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:06.495668 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:06.499258 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0326aa-106a-4c3a-b738-9339ea3358b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:06.500693 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0326aa-106a-4c3a-b738-9339ea3358b9 service neutron] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:06.511025 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 745/1521] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:08:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:08:06.966719 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f841e8ea-6912-4a51-b14a-14ac92c440f7 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:06.968327 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-f841e8ea-6912-4a51-b14a-14ac92c440f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:06.981609 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f841e8ea-6912-4a51-b14a-14ac92c440f7 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:06.986677 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b82e56-f7e5-44a2-a19f-3283c68c6cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:06.987504 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b82e56-f7e5-44a2-a19f-3283c68c6cb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:06.987797 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b82e56-f7e5-44a2-a19f-3283c68c6cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:06.988069 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b82e56-f7e5-44a2-a19f-3283c68c6cb7 admin admin] PATH_INFO: `/v3/projects/1480461d859a423fb589e6baa6438d81` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:06.990245 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b82e56-f7e5-44a2-a19f-3283c68c6cb7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1480461d859a423fb589e6baa6438d81)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:06.991175 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b82e56-f7e5-44a2-a19f-3283c68c6cb7 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:07.062697 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:07.075793 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-63b82e56-f7e5-44a2-a19f-3283c68c6cb7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1480461d859a423fb589e6baa6438d81', 'request_id': 'req-63b82e56-f7e5-44a2-a19f-3283c68c6cb7'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:07.082040 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-63b82e56-f7e5-44a2-a19f-3283c68c6cb7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1480461d859a423fb589e6baa6438d81', 'request_id': 'req-63b82e56-f7e5-44a2-a19f-3283c68c6cb7'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:07.130680 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-63b82e56-f7e5-44a2-a19f-3283c68c6cb7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1480461d859a423fb589e6baa6438d81', 'request_id': 'req-63b82e56-f7e5-44a2-a19f-3283c68c6cb7'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:07.142136 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-63b82e56-f7e5-44a2-a19f-3283c68c6cb7 admin admin] The token cache is being invalidate because project 1480461d859a423fb589e6baa6438d81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81517) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:08:07.142705 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-63b82e56-f7e5-44a2-a19f-3283c68c6cb7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:07.143838 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-63b82e56-f7e5-44a2-a19f-3283c68c6cb7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:07.145398 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 777/1522] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:08:06 2022] DELETE /identity/v3/projects/1480461d859a423fb589e6baa6438d81 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:08:07.149253 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0326aa-106a-4c3a-b738-9339ea3358b9 service neutron] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:07.160628 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0326aa-106a-4c3a-b738-9339ea3358b9 service neutron] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:07.216298 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b571e34-1cbe-4f38-8cb9-37436ca8d68e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:07.217826 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b571e34-1cbe-4f38-8cb9-37436ca8d68e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:07.217826 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b571e34-1cbe-4f38-8cb9-37436ca8d68e admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:07.217826 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b571e34-1cbe-4f38-8cb9-37436ca8d68e admin admin] PATH_INFO: `/v3/users/eb00b9de1cce48a093b7de2bfe2adbd0` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:07.218776 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b571e34-1cbe-4f38-8cb9-37436ca8d68e admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb00b9de1cce48a093b7de2bfe2adbd0)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:07.220826 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b571e34-1cbe-4f38-8cb9-37436ca8d68e admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:07.413850 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-0b571e34-1cbe-4f38-8cb9-37436ca8d68e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb00b9de1cce48a093b7de2bfe2adbd0'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:07.424701 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-0b571e34-1cbe-4f38-8cb9-37436ca8d68e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb00b9de1cce48a093b7de2bfe2adbd0'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:07.440171 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-0b571e34-1cbe-4f38-8cb9-37436ca8d68e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb00b9de1cce48a093b7de2bfe2adbd0'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:07.440930 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-0b571e34-1cbe-4f38-8cb9-37436ca8d68e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb00b9de1cce48a093b7de2bfe2adbd0'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:07.542239 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-0b571e34-1cbe-4f38-8cb9-37436ca8d68e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb00b9de1cce48a093b7de2bfe2adbd0'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:07.552245 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 746/1523] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:08:07 2022] DELETE /identity/v3/users/eb00b9de1cce48a093b7de2bfe2adbd0 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:08:08.116281 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b82e56-f7e5-44a2-a19f-3283c68c6cb7 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:08.119924 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-63b82e56-f7e5-44a2-a19f-3283c68c6cb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:08.179079 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b82e56-f7e5-44a2-a19f-3283c68c6cb7 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:08.279042 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd70366-1234-444e-af34-bf5b3a12af7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:08.279698 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd70366-1234-444e-af34-bf5b3a12af7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:08.280447 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd70366-1234-444e-af34-bf5b3a12af7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:08.280447 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd70366-1234-444e-af34-bf5b3a12af7b admin admin] PATH_INFO: `/v3/projects/e68fd16aec1f444293ba24772452ed5b` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:08.281547 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd70366-1234-444e-af34-bf5b3a12af7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e68fd16aec1f444293ba24772452ed5b)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:08.283700 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd70366-1234-444e-af34-bf5b3a12af7b admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:08.383945 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-6dd70366-1234-444e-af34-bf5b3a12af7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e68fd16aec1f444293ba24772452ed5b', 'request_id': 'req-6dd70366-1234-444e-af34-bf5b3a12af7b'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:08.391002 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-6dd70366-1234-444e-af34-bf5b3a12af7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e68fd16aec1f444293ba24772452ed5b', 'request_id': 'req-6dd70366-1234-444e-af34-bf5b3a12af7b'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:08.445666 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-6dd70366-1234-444e-af34-bf5b3a12af7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e68fd16aec1f444293ba24772452ed5b', 'request_id': 'req-6dd70366-1234-444e-af34-bf5b3a12af7b'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:08.468855 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-6dd70366-1234-444e-af34-bf5b3a12af7b admin admin] The token cache is being invalidate because project e68fd16aec1f444293ba24772452ed5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81517) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:08:08.468855 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-6dd70366-1234-444e-af34-bf5b3a12af7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:08.468855 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-6dd70366-1234-444e-af34-bf5b3a12af7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:08.474374 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 778/1524] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:08:08 2022] DELETE /identity/v3/projects/e68fd16aec1f444293ba24772452ed5b => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:08:08.476046 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d309fd4d-d5c5-4ed7-a097-088813bfc1ca None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:08.476526 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d309fd4d-d5c5-4ed7-a097-088813bfc1ca None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:08.476672 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d309fd4d-d5c5-4ed7-a097-088813bfc1ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:08.516282 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-d309fd4d-d5c5-4ed7-a097-088813bfc1ca None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:08:08.517626 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-d309fd4d-d5c5-4ed7-a097-088813bfc1ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:08.616243 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 747/1525] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:08:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:08:08.626399 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fe11ee-b33b-495a-906d-31d05ce878e0 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:08.627136 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fe11ee-b33b-495a-906d-31d05ce878e0 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:08.627830 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fe11ee-b33b-495a-906d-31d05ce878e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:08.660347 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-f0fe11ee-b33b-495a-906d-31d05ce878e0 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:08:08.660347 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-f0fe11ee-b33b-495a-906d-31d05ce878e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:08.668887 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 779/1526] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:08:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:08:08.673245 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d309fd4d-d5c5-4ed7-a097-088813bfc1ca None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:08.683428 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d309fd4d-d5c5-4ed7-a097-088813bfc1ca None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:08.688184 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8321dfd-6f19-408c-8bf6-f2673f7a0249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:08.690126 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8321dfd-6f19-408c-8bf6-f2673f7a0249 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:08.690126 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8321dfd-6f19-408c-8bf6-f2673f7a0249 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:08.690126 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8321dfd-6f19-408c-8bf6-f2673f7a0249 admin admin] PATH_INFO: `/v3/domains` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:08.690126 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8321dfd-6f19-408c-8bf6-f2673f7a0249 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 03:08:08.690441 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8321dfd-6f19-408c-8bf6-f2673f7a0249 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:08.691911 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8321dfd-6f19-408c-8bf6-f2673f7a0249 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:08.702363 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:08.709060 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 748/1527] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Oct 14 03:08:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 03:08:08.710805 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fe11ee-b33b-495a-906d-31d05ce878e0 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:08.721213 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fe11ee-b33b-495a-906d-31d05ce878e0 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:08.726438 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0797abc2-03dc-498d-ad5f-b39be3006f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:08.727180 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0797abc2-03dc-498d-ad5f-b39be3006f21 admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:08.727521 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0797abc2-03dc-498d-ad5f-b39be3006f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:08.727867 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0797abc2-03dc-498d-ad5f-b39be3006f21 admin admin] PATH_INFO: `/v3/projects` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:08.728582 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0797abc2-03dc-498d-ad5f-b39be3006f21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:08.730822 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0797abc2-03dc-498d-ad5f-b39be3006f21 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:08.836720 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 780/1528] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:08:08 2022] POST /identity/v3/projects => generated 399 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:08:08.840488 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8321dfd-6f19-408c-8bf6-f2673f7a0249 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:08.852805 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8321dfd-6f19-408c-8bf6-f2673f7a0249 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:08.967923 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1339ec22-a4a3-406c-be5c-1f485e03883e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:08.969243 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1339ec22-a4a3-406c-be5c-1f485e03883e admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:08.970604 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1339ec22-a4a3-406c-be5c-1f485e03883e admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:08.971527 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1339ec22-a4a3-406c-be5c-1f485e03883e admin admin] PATH_INFO: `/v3/users` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:08.974990 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1339ec22-a4a3-406c-be5c-1f485e03883e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:08.979014 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1339ec22-a4a3-406c-be5c-1f485e03883e admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:09.064562 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 749/1529] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:08:08 2022] POST /identity/v3/users => generated 300 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:08:09.068383 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0797abc2-03dc-498d-ad5f-b39be3006f21 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:09.082631 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0797abc2-03dc-498d-ad5f-b39be3006f21 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:09.088864 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b0fb4b-bf73-4171-a56d-71a6b3150535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:09.089970 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b0fb4b-bf73-4171-a56d-71a6b3150535 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:09.090329 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b0fb4b-bf73-4171-a56d-71a6b3150535 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:09.090678 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b0fb4b-bf73-4171-a56d-71a6b3150535 admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:09.091441 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b0fb4b-bf73-4171-a56d-71a6b3150535 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:09.093249 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b0fb4b-bf73-4171-a56d-71a6b3150535 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:09.108304 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 781/1530] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:08:09 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:08:09.117443 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1339ec22-a4a3-406c-be5c-1f485e03883e admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:09.130108 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1339ec22-a4a3-406c-be5c-1f485e03883e admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:09.146556 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b219b0f2-f679-43f3-b090-065dd56a4137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:09.148816 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b219b0f2-f679-43f3-b090-065dd56a4137 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:09.148816 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b219b0f2-f679-43f3-b090-065dd56a4137 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:09.148816 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b219b0f2-f679-43f3-b090-065dd56a4137 admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:09.148816 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b219b0f2-f679-43f3-b090-065dd56a4137 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:09.150975 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b219b0f2-f679-43f3-b090-065dd56a4137 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:09.201635 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 750/1531] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:08:09 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:08:09.205917 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b0fb4b-bf73-4171-a56d-71a6b3150535 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:09.227989 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b0fb4b-bf73-4171-a56d-71a6b3150535 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:09.256880 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ca80eb-dd2f-46a5-8865-a304a3222040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:09.257931 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ca80eb-dd2f-46a5-8865-a304a3222040 admin admin] REQUEST_METHOD: `PUT` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:09.258586 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ca80eb-dd2f-46a5-8865-a304a3222040 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:09.259003 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ca80eb-dd2f-46a5-8865-a304a3222040 admin admin] PATH_INFO: `/v3/projects/2f5763faebd649fe92aa51027a30a27d/users/338bc94ab85848698c874cbbd73255f1/roles/9bd0470b544240209a420f640c79c06c` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:09.264821 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:09.281379 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ca80eb-dd2f-46a5-8865-a304a3222040 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f5763faebd649fe92aa51027a30a27d, user_id=338bc94ab85848698c874cbbd73255f1, role_id=9bd0470b544240209a420f640c79c06c)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:09.284643 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ca80eb-dd2f-46a5-8865-a304a3222040 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:09.383849 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 782/1532] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:08:09 2022] PUT /identity/v3/projects/2f5763faebd649fe92aa51027a30a27d/users/338bc94ab85848698c874cbbd73255f1/roles/9bd0470b544240209a420f640c79c06c => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:08:09.396200 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c1898b-51f8-4455-8d65-b6682c3fe9c8 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:09.396489 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c1898b-51f8-4455-8d65-b6682c3fe9c8 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:09.396770 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c1898b-51f8-4455-8d65-b6682c3fe9c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:09.446487 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-62c1898b-51f8-4455-8d65-b6682c3fe9c8 None None] MFA Rules not processed for user `338bc94ab85848698c874cbbd73255f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:08:09.446487 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-62c1898b-51f8-4455-8d65-b6682c3fe9c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:09.481399 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ca80eb-dd2f-46a5-8865-a304a3222040 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:09.483718 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-59ca80eb-dd2f-46a5-8865-a304a3222040 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:09.562601 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 751/1533] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:08:09 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:08:09.574251 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c1898b-51f8-4455-8d65-b6682c3fe9c8 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:09.592879 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c1898b-51f8-4455-8d65-b6682c3fe9c8 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:09.599969 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ca80eb-dd2f-46a5-8865-a304a3222040 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:09.604393 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0f8b92-286c-4c31-900d-2301e1512f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:09.604995 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0f8b92-286c-4c31-900d-2301e1512f3b admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:09.606035 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0f8b92-286c-4c31-900d-2301e1512f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:09.606475 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0f8b92-286c-4c31-900d-2301e1512f3b admin admin] PATH_INFO: `/v3/projects` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:09.608322 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0f8b92-286c-4c31-900d-2301e1512f3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:09.611362 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0f8b92-286c-4c31-900d-2301e1512f3b admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:09.619939 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a6308a-05e7-4deb-b9f5-35ee731add29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:09.620804 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a6308a-05e7-4deb-b9f5-35ee731add29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:09.621217 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a6308a-05e7-4deb-b9f5-35ee731add29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:09.621640 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a6308a-05e7-4deb-b9f5-35ee731add29 admin admin] PATH_INFO: `/v3/users/7f80dd0aaf2642db89dabbc4951a917f` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:09.624931 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a6308a-05e7-4deb-b9f5-35ee731add29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f80dd0aaf2642db89dabbc4951a917f)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:09.641104 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a6308a-05e7-4deb-b9f5-35ee731add29 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:09.775341 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 752/1534] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:08:09 2022] POST /identity/v3/projects => generated 401 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:08:09.776445 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0f8b92-286c-4c31-900d-2301e1512f3b admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:09.853775 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:09.877918 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-50a6308a-05e7-4deb-b9f5-35ee731add29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f80dd0aaf2642db89dabbc4951a917f'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:09.879099 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-50a6308a-05e7-4deb-b9f5-35ee731add29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f80dd0aaf2642db89dabbc4951a917f'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:09.888643 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0f8b92-286c-4c31-900d-2301e1512f3b admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:09.898005 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebd113c-b673-4c17-86be-506c3825fc2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:09.899217 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebd113c-b673-4c17-86be-506c3825fc2e admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:09.899661 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebd113c-b673-4c17-86be-506c3825fc2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:09.899954 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebd113c-b673-4c17-86be-506c3825fc2e admin admin] PATH_INFO: `/v3/users` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:09.900847 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebd113c-b673-4c17-86be-506c3825fc2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:09.902736 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebd113c-b673-4c17-86be-506c3825fc2e admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:10.025937 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-50a6308a-05e7-4deb-b9f5-35ee731add29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f80dd0aaf2642db89dabbc4951a917f'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:10.025937 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-50a6308a-05e7-4deb-b9f5-35ee731add29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f80dd0aaf2642db89dabbc4951a917f'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:10.025937 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-50a6308a-05e7-4deb-b9f5-35ee731add29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f80dd0aaf2642db89dabbc4951a917f'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:10.039262 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 783/1535] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:08:09 2022] DELETE /identity/v3/users/7f80dd0aaf2642db89dabbc4951a917f => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:08:10.052039 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5de715-1976-483b-9f35-c1a12796764d None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:10.080922 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5de715-1976-483b-9f35-c1a12796764d None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:10.080922 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5de715-1976-483b-9f35-c1a12796764d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:10.080922 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-da5de715-1976-483b-9f35-c1a12796764d None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:08:10.080922 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-da5de715-1976-483b-9f35-c1a12796764d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:10.105673 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 753/1536] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:08:09 2022] POST /identity/v3/users => generated 307 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 03:08:10.109733 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebd113c-b673-4c17-86be-506c3825fc2e admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:10.112198 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-eebd113c-b673-4c17-86be-506c3825fc2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:10.128411 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebd113c-b673-4c17-86be-506c3825fc2e admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:10.153458 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 784/1537] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:08:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:08:10.163919 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5de715-1976-483b-9f35-c1a12796764d None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:10.166281 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-da5de715-1976-483b-9f35-c1a12796764d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:10.198901 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85da82c7-5f90-4c23-8d12-66c2d601ec83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:10.199515 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85da82c7-5f90-4c23-8d12-66c2d601ec83 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:10.199800 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85da82c7-5f90-4c23-8d12-66c2d601ec83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:10.200080 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85da82c7-5f90-4c23-8d12-66c2d601ec83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:10.200727 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85da82c7-5f90-4c23-8d12-66c2d601ec83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:10.202385 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85da82c7-5f90-4c23-8d12-66c2d601ec83 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:10.213945 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 754/1538] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:08:10 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:08:10.218208 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85da82c7-5f90-4c23-8d12-66c2d601ec83 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:10.230351 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85da82c7-5f90-4c23-8d12-66c2d601ec83 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:10.235541 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74888b3-a767-48aa-8ec7-8dd65aaed9e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:10.236233 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74888b3-a767-48aa-8ec7-8dd65aaed9e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:10.236512 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5de715-1976-483b-9f35-c1a12796764d None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:10.236983 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74888b3-a767-48aa-8ec7-8dd65aaed9e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:10.236983 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74888b3-a767-48aa-8ec7-8dd65aaed9e8 admin admin] PATH_INFO: `/v3/projects/d33a6ff5d3e3467eb0b4d82a5bde15ce/users/7add678281f046b8b5c7c58bb90fbb55/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:10.240263 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13fc680-a4e9-4888-8f55-c3ce39a38c4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:10.240868 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13fc680-a4e9-4888-8f55-c3ce39a38c4e service neutron] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:10.241150 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13fc680-a4e9-4888-8f55-c3ce39a38c4e service neutron] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:10.241444 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13fc680-a4e9-4888-8f55-c3ce39a38c4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:10.251432 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74888b3-a767-48aa-8ec7-8dd65aaed9e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d33a6ff5d3e3467eb0b4d82a5bde15ce, user_id=7add678281f046b8b5c7c58bb90fbb55, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:10.251581 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13fc680-a4e9-4888-8f55-c3ce39a38c4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:10.253047 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13fc680-a4e9-4888-8f55-c3ce39a38c4e service neutron] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:10.255847 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74888b3-a767-48aa-8ec7-8dd65aaed9e8 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:10.264575 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 785/1539] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:08:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 03:08:10.466385 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 755/1540] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:08:10 2022] PUT /identity/v3/projects/d33a6ff5d3e3467eb0b4d82a5bde15ce/users/7add678281f046b8b5c7c58bb90fbb55/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 03:08:10.472596 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13fc680-a4e9-4888-8f55-c3ce39a38c4e service neutron] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:10.494613 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13fc680-a4e9-4888-8f55-c3ce39a38c4e service neutron] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:10.559310 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85712909-3fed-42ad-a740-fa94a83c191d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:10.560612 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85712909-3fed-42ad-a740-fa94a83c191d admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:10.560967 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85712909-3fed-42ad-a740-fa94a83c191d admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:10.561311 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85712909-3fed-42ad-a740-fa94a83c191d admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:10.562426 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85712909-3fed-42ad-a740-fa94a83c191d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:10.564278 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85712909-3fed-42ad-a740-fa94a83c191d admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:10.580930 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 786/1541] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:08:10 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:08:10.581722 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74888b3-a767-48aa-8ec7-8dd65aaed9e8 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:10.593668 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74888b3-a767-48aa-8ec7-8dd65aaed9e8 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:10.609282 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb09104e-a780-4ac3-ba4d-9f72ef2fbca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:10.610568 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb09104e-a780-4ac3-ba4d-9f72ef2fbca8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:10.610850 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb09104e-a780-4ac3-ba4d-9f72ef2fbca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:10.611152 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb09104e-a780-4ac3-ba4d-9f72ef2fbca8 admin admin] PATH_INFO: `/v3/projects/d33a6ff5d3e3467eb0b4d82a5bde15ce/users/7add678281f046b8b5c7c58bb90fbb55/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:10.615963 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb09104e-a780-4ac3-ba4d-9f72ef2fbca8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d33a6ff5d3e3467eb0b4d82a5bde15ce, user_id=7add678281f046b8b5c7c58bb90fbb55, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:10.620411 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb09104e-a780-4ac3-ba4d-9f72ef2fbca8 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:10.633978 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 756/1542] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:08:10 2022] PUT /identity/v3/projects/d33a6ff5d3e3467eb0b4d82a5bde15ce/users/7add678281f046b8b5c7c58bb90fbb55/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 03:08:10.645411 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420090d7-fc4a-4ea0-abd7-eb5bfa7a3067 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:10.645695 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420090d7-fc4a-4ea0-abd7-eb5bfa7a3067 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:10.646019 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420090d7-fc4a-4ea0-abd7-eb5bfa7a3067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:10.686743 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-420090d7-fc4a-4ea0-abd7-eb5bfa7a3067 None None] MFA Rules not processed for user `7add678281f046b8b5c7c58bb90fbb55`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:08:10.688618 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-420090d7-fc4a-4ea0-abd7-eb5bfa7a3067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:10.780672 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 787/1543] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:08:10 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:08:10.789048 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb09104e-a780-4ac3-ba4d-9f72ef2fbca8 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:10.791115 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-cb09104e-a780-4ac3-ba4d-9f72ef2fbca8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:10.823129 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:10.863802 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb09104e-a780-4ac3-ba4d-9f72ef2fbca8 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:10.875878 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6eb08f-f354-493a-96d9-05cd1b4e1cf7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab599d98b0624a2e89090312bcab713f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:10.877526 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6eb08f-f354-493a-96d9-05cd1b4e1cf7 service cinder] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:10.878117 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6eb08f-f354-493a-96d9-05cd1b4e1cf7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:10.878594 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6eb08f-f354-493a-96d9-05cd1b4e1cf7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:10.897808 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6eb08f-f354-493a-96d9-05cd1b4e1cf7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:10.897808 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6eb08f-f354-493a-96d9-05cd1b4e1cf7 service cinder] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:10.908433 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 757/1544] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:08:10 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:08:11.107251 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420090d7-fc4a-4ea0-abd7-eb5bfa7a3067 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:11.120077 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420090d7-fc4a-4ea0-abd7-eb5bfa7a3067 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:11.125061 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83418416-98ad-4701-b179-0cbb4dc9a578 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab599d98b0624a2e89090312bcab713f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:11.127411 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83418416-98ad-4701-b179-0cbb4dc9a578 service cinder] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:11.127935 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83418416-98ad-4701-b179-0cbb4dc9a578 service cinder] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:11.128412 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83418416-98ad-4701-b179-0cbb4dc9a578 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:11.130931 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-83418416-98ad-4701-b179-0cbb4dc9a578 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:11.211940 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83418416-98ad-4701-b179-0cbb4dc9a578 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:11.213713 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83418416-98ad-4701-b179-0cbb4dc9a578 service cinder] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:11.231694 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 788/1545] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:08:11 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:08:11.245864 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6eb08f-f354-493a-96d9-05cd1b4e1cf7 service cinder] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:11.247357 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-4d6eb08f-f354-493a-96d9-05cd1b4e1cf7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:11.334105 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:11.397285 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6eb08f-f354-493a-96d9-05cd1b4e1cf7 service cinder] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:11.401845 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57fad4b-0716-46b3-9bb6-f1efa797be09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:11.402784 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57fad4b-0716-46b3-9bb6-f1efa797be09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:11.403140 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57fad4b-0716-46b3-9bb6-f1efa797be09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:11.403641 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57fad4b-0716-46b3-9bb6-f1efa797be09 admin admin] PATH_INFO: `/v3/projects/bf74e7fe6f2d49d4adbae394b68be44b` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:11.405076 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57fad4b-0716-46b3-9bb6-f1efa797be09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf74e7fe6f2d49d4adbae394b68be44b)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:11.407150 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57fad4b-0716-46b3-9bb6-f1efa797be09 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:11.568498 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-f57fad4b-0716-46b3-9bb6-f1efa797be09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf74e7fe6f2d49d4adbae394b68be44b', 'request_id': 'req-f57fad4b-0716-46b3-9bb6-f1efa797be09'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:11.578000 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-f57fad4b-0716-46b3-9bb6-f1efa797be09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf74e7fe6f2d49d4adbae394b68be44b', 'request_id': 'req-f57fad4b-0716-46b3-9bb6-f1efa797be09'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:11.667473 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-f57fad4b-0716-46b3-9bb6-f1efa797be09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf74e7fe6f2d49d4adbae394b68be44b', 'request_id': 'req-f57fad4b-0716-46b3-9bb6-f1efa797be09'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:11.689647 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-f57fad4b-0716-46b3-9bb6-f1efa797be09 admin admin] The token cache is being invalidate because project bf74e7fe6f2d49d4adbae394b68be44b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:08:11.690100 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-f57fad4b-0716-46b3-9bb6-f1efa797be09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:11.691339 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-f57fad4b-0716-46b3-9bb6-f1efa797be09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:11.694999 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 758/1546] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:08:11 2022] DELETE /identity/v3/projects/bf74e7fe6f2d49d4adbae394b68be44b => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:08:11.708039 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83418416-98ad-4701-b179-0cbb4dc9a578 service cinder] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:11.722165 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83418416-98ad-4701-b179-0cbb4dc9a578 service cinder] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:11.797456 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1891429d-22c1-45ce-86de-5e9724bbbebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:11.798729 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1891429d-22c1-45ce-86de-5e9724bbbebc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:11.799024 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1891429d-22c1-45ce-86de-5e9724bbbebc admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:11.799364 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1891429d-22c1-45ce-86de-5e9724bbbebc admin admin] PATH_INFO: `/v3/users/df5d9a3a1d084be8bda8eebb1c53bf00` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:11.801380 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1891429d-22c1-45ce-86de-5e9724bbbebc admin admin] RBAC: Authorizing `identity:delete_user(user_id=df5d9a3a1d084be8bda8eebb1c53bf00)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:11.804195 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1891429d-22c1-45ce-86de-5e9724bbbebc admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:12.037737 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-1891429d-22c1-45ce-86de-5e9724bbbebc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df5d9a3a1d084be8bda8eebb1c53bf00'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:12.038585 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-1891429d-22c1-45ce-86de-5e9724bbbebc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df5d9a3a1d084be8bda8eebb1c53bf00'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:12.108673 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-1891429d-22c1-45ce-86de-5e9724bbbebc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df5d9a3a1d084be8bda8eebb1c53bf00'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:12.109123 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-1891429d-22c1-45ce-86de-5e9724bbbebc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df5d9a3a1d084be8bda8eebb1c53bf00'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:12.137747 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-1891429d-22c1-45ce-86de-5e9724bbbebc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df5d9a3a1d084be8bda8eebb1c53bf00'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:12.153263 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 789/1547] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:08:11 2022] DELETE /identity/v3/users/df5d9a3a1d084be8bda8eebb1c53bf00 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:08:12.606094 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:12.817899 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57fad4b-0716-46b3-9bb6-f1efa797be09 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:12.824745 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-f57fad4b-0716-46b3-9bb6-f1efa797be09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:12.833367 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57fad4b-0716-46b3-9bb6-f1efa797be09 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:12.905568 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab87862-a080-42f5-b61e-ea680dc1861a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:12.906501 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab87862-a080-42f5-b61e-ea680dc1861a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:12.911383 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab87862-a080-42f5-b61e-ea680dc1861a admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:12.911383 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab87862-a080-42f5-b61e-ea680dc1861a admin admin] PATH_INFO: `/v3/projects/38b83be5aa60429fb8a69bd5d2f6b8dc` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:12.911383 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab87862-a080-42f5-b61e-ea680dc1861a admin admin] RBAC: Authorizing `identity:delete_project(project_id=38b83be5aa60429fb8a69bd5d2f6b8dc)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:12.918814 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab87862-a080-42f5-b61e-ea680dc1861a admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:13.122019 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-bab87862-a080-42f5-b61e-ea680dc1861a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38b83be5aa60429fb8a69bd5d2f6b8dc', 'request_id': 'req-bab87862-a080-42f5-b61e-ea680dc1861a'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:13.131590 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-bab87862-a080-42f5-b61e-ea680dc1861a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38b83be5aa60429fb8a69bd5d2f6b8dc', 'request_id': 'req-bab87862-a080-42f5-b61e-ea680dc1861a'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:13.233678 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-bab87862-a080-42f5-b61e-ea680dc1861a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38b83be5aa60429fb8a69bd5d2f6b8dc', 'request_id': 'req-bab87862-a080-42f5-b61e-ea680dc1861a'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:13.244743 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:13.279498 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-bab87862-a080-42f5-b61e-ea680dc1861a admin admin] The token cache is being invalidate because project 38b83be5aa60429fb8a69bd5d2f6b8dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:08:13.280109 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-bab87862-a080-42f5-b61e-ea680dc1861a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:13.287230 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-bab87862-a080-42f5-b61e-ea680dc1861a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:13.293530 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 759/1548] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:08:12 2022] DELETE /identity/v3/projects/38b83be5aa60429fb8a69bd5d2f6b8dc => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 03:08:13.298389 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b103034f-62f3-4780-9db2-2655c457c82a None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:13.299233 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b103034f-62f3-4780-9db2-2655c457c82a None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:13.299674 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b103034f-62f3-4780-9db2-2655c457c82a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:13.336144 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-b103034f-62f3-4780-9db2-2655c457c82a None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:08:13.348567 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-b103034f-62f3-4780-9db2-2655c457c82a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:13.448568 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:13.482747 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 790/1549] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:08:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 03:08:13.484519 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943b8b26-97a1-4387-9d5f-4e51b2b23a3e None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:13.484902 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943b8b26-97a1-4387-9d5f-4e51b2b23a3e None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:13.485251 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943b8b26-97a1-4387-9d5f-4e51b2b23a3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:13.517426 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-943b8b26-97a1-4387-9d5f-4e51b2b23a3e None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:08:13.519503 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-943b8b26-97a1-4387-9d5f-4e51b2b23a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:13.538298 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 760/1550] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:08:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 03:08:13.540356 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b103034f-62f3-4780-9db2-2655c457c82a None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:13.551704 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b103034f-62f3-4780-9db2-2655c457c82a None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:13.557643 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a2a53c-9457-44a0-865a-83502e5f87ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:13.558526 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a2a53c-9457-44a0-865a-83502e5f87ff admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:13.558885 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a2a53c-9457-44a0-865a-83502e5f87ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:13.559237 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a2a53c-9457-44a0-865a-83502e5f87ff admin admin] PATH_INFO: `/v3/domains` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:13.560187 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a2a53c-9457-44a0-865a-83502e5f87ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 03:08:13.560587 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a2a53c-9457-44a0-865a-83502e5f87ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:13.562351 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a2a53c-9457-44a0-865a-83502e5f87ff admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:13.588667 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 791/1551] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Oct 14 03:08:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 03:08:13.593297 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943b8b26-97a1-4387-9d5f-4e51b2b23a3e None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:13.606435 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943b8b26-97a1-4387-9d5f-4e51b2b23a3e None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:13.619392 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea27427e-924d-414e-9e39-209ed4cfa8bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:13.620031 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea27427e-924d-414e-9e39-209ed4cfa8bf admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:13.620318 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea27427e-924d-414e-9e39-209ed4cfa8bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:13.622854 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea27427e-924d-414e-9e39-209ed4cfa8bf admin admin] PATH_INFO: `/v3/projects` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:13.623550 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea27427e-924d-414e-9e39-209ed4cfa8bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:13.626211 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea27427e-924d-414e-9e39-209ed4cfa8bf admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:13.774514 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 761/1552] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:08:13 2022] POST /identity/v3/projects => generated 407 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:08:13.778878 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a2a53c-9457-44a0-865a-83502e5f87ff admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:13.791632 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a2a53c-9457-44a0-865a-83502e5f87ff admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:13.861273 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689a2e70-33a3-4dce-997f-1ba45ba238d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:13.862644 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a2e70-33a3-4dce-997f-1ba45ba238d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:13.862644 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a2e70-33a3-4dce-997f-1ba45ba238d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:13.862801 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a2e70-33a3-4dce-997f-1ba45ba238d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:13.863309 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689a2e70-33a3-4dce-997f-1ba45ba238d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:13.865247 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689a2e70-33a3-4dce-997f-1ba45ba238d9 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:13.973797 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 792/1553] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:08:13 2022] POST /identity/v3/users => generated 304 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:08:13.975584 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea27427e-924d-414e-9e39-209ed4cfa8bf admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:13.987319 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea27427e-924d-414e-9e39-209ed4cfa8bf admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:13.992146 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf15b8c-f2d3-4e15-958f-c8d9bb387fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:13.993285 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf15b8c-f2d3-4e15-958f-c8d9bb387fd3 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:13.993535 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf15b8c-f2d3-4e15-958f-c8d9bb387fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:13.993841 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf15b8c-f2d3-4e15-958f-c8d9bb387fd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:13.994485 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf15b8c-f2d3-4e15-958f-c8d9bb387fd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:13.996120 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf15b8c-f2d3-4e15-958f-c8d9bb387fd3 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:14.006914 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 762/1554] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:08:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 03:08:14.015227 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689a2e70-33a3-4dce-997f-1ba45ba238d9 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:14.025647 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689a2e70-33a3-4dce-997f-1ba45ba238d9 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:14.030504 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5b731c-34c7-46ac-b9e2-049502403de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:14.031104 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b731c-34c7-46ac-b9e2-049502403de8 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:14.031395 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b731c-34c7-46ac-b9e2-049502403de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:14.031674 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b731c-34c7-46ac-b9e2-049502403de8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:14.032326 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5b731c-34c7-46ac-b9e2-049502403de8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:14.033971 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5b731c-34c7-46ac-b9e2-049502403de8 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:14.044684 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 793/1555] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:08:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:08:14.049139 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf15b8c-f2d3-4e15-958f-c8d9bb387fd3 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:14.060780 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf15b8c-f2d3-4e15-958f-c8d9bb387fd3 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:14.065686 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893843d2-9380-47f7-9fb6-14e9c68ba00b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:14.066388 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893843d2-9380-47f7-9fb6-14e9c68ba00b admin admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:14.066672 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893843d2-9380-47f7-9fb6-14e9c68ba00b admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:14.066952 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893843d2-9380-47f7-9fb6-14e9c68ba00b admin admin] PATH_INFO: `/v3/projects/d2a7863d31e04296b9ee7b84ed92780b/users/cd1001992c314b6baf2d00552eb930a8/roles/9bd0470b544240209a420f640c79c06c` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:14.089962 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893843d2-9380-47f7-9fb6-14e9c68ba00b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2a7863d31e04296b9ee7b84ed92780b, user_id=cd1001992c314b6baf2d00552eb930a8, role_id=9bd0470b544240209a420f640c79c06c)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:14.093189 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893843d2-9380-47f7-9fb6-14e9c68ba00b admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:14.232065 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 763/1556] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:08:14 2022] PUT /identity/v3/projects/d2a7863d31e04296b9ee7b84ed92780b/users/cd1001992c314b6baf2d00552eb930a8/roles/9bd0470b544240209a420f640c79c06c => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 03:08:14.232850 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b7e581-9d1d-4ce8-bd1c-d238a8a6e2d4 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:14.233141 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b7e581-9d1d-4ce8-bd1c-d238a8a6e2d4 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:14.233422 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b7e581-9d1d-4ce8-bd1c-d238a8a6e2d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:14.288951 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-66b7e581-9d1d-4ce8-bd1c-d238a8a6e2d4 None None] MFA Rules not processed for user `cd1001992c314b6baf2d00552eb930a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:08:14.290261 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-66b7e581-9d1d-4ce8-bd1c-d238a8a6e2d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:14.370882 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 794/1557] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:08:14 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:08:14.375167 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893843d2-9380-47f7-9fb6-14e9c68ba00b admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:14.469502 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893843d2-9380-47f7-9fb6-14e9c68ba00b admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:14.475417 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e48393e-d907-43bc-b8cf-305097347b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:14.476046 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e48393e-d907-43bc-b8cf-305097347b56 admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:14.476328 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e48393e-d907-43bc-b8cf-305097347b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:14.477151 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e48393e-d907-43bc-b8cf-305097347b56 admin admin] PATH_INFO: `/v3/projects` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:14.478205 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e48393e-d907-43bc-b8cf-305097347b56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:14.480486 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e48393e-d907-43bc-b8cf-305097347b56 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:14.605111 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b7e581-9d1d-4ce8-bd1c-d238a8a6e2d4 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:14.610430 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 764/1558] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:08:14 2022] POST /identity/v3/projects => generated 407 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:08:14.626465 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b7e581-9d1d-4ce8-bd1c-d238a8a6e2d4 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:14.757427 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59bb20a-299f-49cb-9033-756c6988a6ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:14.758149 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59bb20a-299f-49cb-9033-756c6988a6ce admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:14.758433 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59bb20a-299f-49cb-9033-756c6988a6ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:14.758713 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59bb20a-299f-49cb-9033-756c6988a6ce admin admin] PATH_INFO: `/v3/users` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:14.759471 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59bb20a-299f-49cb-9033-756c6988a6ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:14.761862 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59bb20a-299f-49cb-9033-756c6988a6ce admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:14.803012 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e48393e-d907-43bc-b8cf-305097347b56 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:14.805097 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-8e48393e-d907-43bc-b8cf-305097347b56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:14.819688 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e48393e-d907-43bc-b8cf-305097347b56 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:14.825904 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd7069a-8a0a-4176-9721-773349b8fe06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:14.827403 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd7069a-8a0a-4176-9721-773349b8fe06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:14.838406 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd7069a-8a0a-4176-9721-773349b8fe06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:14.838406 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd7069a-8a0a-4176-9721-773349b8fe06 admin admin] PATH_INFO: `/v3/users/469802150c6b4cb886eee2a33f618e2f` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:14.838406 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd7069a-8a0a-4176-9721-773349b8fe06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=469802150c6b4cb886eee2a33f618e2f)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:14.838406 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd7069a-8a0a-4176-9721-773349b8fe06 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:14.865304 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 795/1559] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:08:14 2022] POST /identity/v3/users => generated 310 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 03:08:14.870352 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59bb20a-299f-49cb-9033-756c6988a6ce admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:14.883302 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59bb20a-299f-49cb-9033-756c6988a6ce admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:14.895148 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bc3b90-87b1-40ba-bb9f-45dc3d2a2611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:14.895909 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bc3b90-87b1-40ba-bb9f-45dc3d2a2611 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:14.896255 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bc3b90-87b1-40ba-bb9f-45dc3d2a2611 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:14.896594 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bc3b90-87b1-40ba-bb9f-45dc3d2a2611 admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:14.897318 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bc3b90-87b1-40ba-bb9f-45dc3d2a2611 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:14.899775 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bc3b90-87b1-40ba-bb9f-45dc3d2a2611 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:14.917577 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 796/1560] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:08:14 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:08:14.919550 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bc3b90-87b1-40ba-bb9f-45dc3d2a2611 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:14.932027 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bc3b90-87b1-40ba-bb9f-45dc3d2a2611 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:14.947644 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db6bc50-1351-409d-98d5-715bdad16dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:14.948537 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db6bc50-1351-409d-98d5-715bdad16dde admin admin] REQUEST_METHOD: `PUT` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:14.949744 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db6bc50-1351-409d-98d5-715bdad16dde admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:14.949744 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db6bc50-1351-409d-98d5-715bdad16dde admin admin] PATH_INFO: `/v3/projects/f402d8a43b5d435b9ae4d6f6015bfa6f/users/7c00170974ef4ac7b8fd6bc49769b50e/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:14.992753 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db6bc50-1351-409d-98d5-715bdad16dde admin admin] RBAC: Authorizing `identity:create_grant(project_id=f402d8a43b5d435b9ae4d6f6015bfa6f, user_id=7c00170974ef4ac7b8fd6bc49769b50e, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:14.999676 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db6bc50-1351-409d-98d5-715bdad16dde admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:15.114538 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 797/1561] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:08:14 2022] PUT /identity/v3/projects/f402d8a43b5d435b9ae4d6f6015bfa6f/users/7c00170974ef4ac7b8fd6bc49769b50e/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:08:15.118980 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db6bc50-1351-409d-98d5-715bdad16dde admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:15.151285 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-bbd7069a-8a0a-4176-9721-773349b8fe06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '469802150c6b4cb886eee2a33f618e2f'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:15.168668 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-bbd7069a-8a0a-4176-9721-773349b8fe06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '469802150c6b4cb886eee2a33f618e2f'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:15.195919 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-bbd7069a-8a0a-4176-9721-773349b8fe06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '469802150c6b4cb886eee2a33f618e2f'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:15.195919 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-bbd7069a-8a0a-4176-9721-773349b8fe06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '469802150c6b4cb886eee2a33f618e2f'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:15.197876 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:15.229886 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db6bc50-1351-409d-98d5-715bdad16dde admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:15.255998 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14948e4-2dc8-462e-935b-356978a21741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:15.258431 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14948e4-2dc8-462e-935b-356978a21741 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:15.258431 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14948e4-2dc8-462e-935b-356978a21741 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:15.258733 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14948e4-2dc8-462e-935b-356978a21741 admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:15.260882 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14948e4-2dc8-462e-935b-356978a21741 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:15.266646 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14948e4-2dc8-462e-935b-356978a21741 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:15.295893 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 798/1562] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:08:15 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:08:15.311817 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14948e4-2dc8-462e-935b-356978a21741 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:15.353851 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-d14948e4-2dc8-462e-935b-356978a21741 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:15.353851 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14948e4-2dc8-462e-935b-356978a21741 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:15.358763 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58e5bed-ebc6-4db4-9cc0-4bfee6fe74a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:15.360247 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58e5bed-ebc6-4db4-9cc0-4bfee6fe74a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:15.360733 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58e5bed-ebc6-4db4-9cc0-4bfee6fe74a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:15.361163 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58e5bed-ebc6-4db4-9cc0-4bfee6fe74a2 admin admin] PATH_INFO: `/v3/projects/f402d8a43b5d435b9ae4d6f6015bfa6f/users/7c00170974ef4ac7b8fd6bc49769b50e/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:15.363923 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58e5bed-ebc6-4db4-9cc0-4bfee6fe74a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f402d8a43b5d435b9ae4d6f6015bfa6f, user_id=7c00170974ef4ac7b8fd6bc49769b50e, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:15.367888 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58e5bed-ebc6-4db4-9cc0-4bfee6fe74a2 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:15.397851 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 799/1563] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:08:15 2022] PUT /identity/v3/projects/f402d8a43b5d435b9ae4d6f6015bfa6f/users/7c00170974ef4ac7b8fd6bc49769b50e/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 03:08:15.404671 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b23486-357c-4974-a877-7f1446305be2 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:15.405082 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b23486-357c-4974-a877-7f1446305be2 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:15.405462 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b23486-357c-4974-a877-7f1446305be2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:15.493663 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-54b23486-357c-4974-a877-7f1446305be2 None None] MFA Rules not processed for user `7c00170974ef4ac7b8fd6bc49769b50e`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:08:15.495110 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-54b23486-357c-4974-a877-7f1446305be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:15.634513 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-bbd7069a-8a0a-4176-9721-773349b8fe06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '469802150c6b4cb886eee2a33f618e2f'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:15.647824 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 765/1564] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:08:14 2022] DELETE /identity/v3/users/469802150c6b4cb886eee2a33f618e2f => generated 0 bytes in 842 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:08:15.649787 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e32130-7c8b-41db-99cb-ed542c12143f None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:15.650234 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e32130-7c8b-41db-99cb-ed542c12143f None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:15.650670 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e32130-7c8b-41db-99cb-ed542c12143f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:15.688729 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-a7e32130-7c8b-41db-99cb-ed542c12143f None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:08:15.703987 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 800/1565] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:08:15 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 03:08:15.704391 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-a7e32130-7c8b-41db-99cb-ed542c12143f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:15.720573 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b23486-357c-4974-a877-7f1446305be2 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:15.723305 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-54b23486-357c-4974-a877-7f1446305be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:15.840730 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 766/1566] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:08:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:08:15.845033 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b23486-357c-4974-a877-7f1446305be2 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:15.876228 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e32130-7c8b-41db-99cb-ed542c12143f None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:15.876228 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-a7e32130-7c8b-41db-99cb-ed542c12143f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:15.915635 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4056a043-0b07-426b-92e2-7a4edc121886 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab599d98b0624a2e89090312bcab713f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:15.916625 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4056a043-0b07-426b-92e2-7a4edc121886 service cinder] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:15.916733 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4056a043-0b07-426b-92e2-7a4edc121886 service cinder] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:15.917137 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4056a043-0b07-426b-92e2-7a4edc121886 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:15.964006 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e32130-7c8b-41db-99cb-ed542c12143f None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:15.968861 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a21aea6-d59e-4186-a181-0e8fa830a7d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:15.969624 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a21aea6-d59e-4186-a181-0e8fa830a7d9 service neutron] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:15.970355 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a21aea6-d59e-4186-a181-0e8fa830a7d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:15.970675 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a21aea6-d59e-4186-a181-0e8fa830a7d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:15.986754 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a21aea6-d59e-4186-a181-0e8fa830a7d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:15.988790 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a21aea6-d59e-4186-a181-0e8fa830a7d9 service neutron] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:16.016895 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 767/1567] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:08:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:08:16.073655 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4056a043-0b07-426b-92e2-7a4edc121886 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:16.075154 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4056a043-0b07-426b-92e2-7a4edc121886 service cinder] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:16.085794 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 801/1568] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:08:15 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:08:16.389369 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:16.400569 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8678b64d-d9aa-4a84-a5e0-dac6d96718b3 None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:16.401065 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8678b64d-d9aa-4a84-a5e0-dac6d96718b3 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:16.401166 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8678b64d-d9aa-4a84-a5e0-dac6d96718b3 None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:16.402508 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 768/1569] 10.222.0.39 () {54 vars in 921 bytes} [Fri Oct 14 03:08:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 03:08:16.420197 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc09ed-443f-48d5-92ff-b29fb0f32a73 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:16.420727 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc09ed-443f-48d5-92ff-b29fb0f32a73 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:16.420727 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc09ed-443f-48d5-92ff-b29fb0f32a73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:16.444324 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-8abc09ed-443f-48d5-92ff-b29fb0f32a73 None None] MFA Rules not processed for user `a6b07bf0faca47b28b50990d1720853f`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:08:16.445613 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-8abc09ed-443f-48d5-92ff-b29fb0f32a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:16.556325 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 802/1570] 10.222.0.39 () {58 vars in 1018 bytes} [Fri Oct 14 03:08:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:08:16.580760 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8678b64d-d9aa-4a84-a5e0-dac6d96718b3 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:16.582940 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-8678b64d-d9aa-4a84-a5e0-dac6d96718b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:16.617635 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8678b64d-d9aa-4a84-a5e0-dac6d96718b3 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:16.636113 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31393da5-555f-4753-9a43-453f04b2b295 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b07bf0faca47b28b50990d1720853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:16.636992 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abc09ed-443f-48d5-92ff-b29fb0f32a73 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:16.637134 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31393da5-555f-4753-9a43-453f04b2b295 service nova] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:16.637433 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31393da5-555f-4753-9a43-453f04b2b295 service nova] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:16.637833 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31393da5-555f-4753-9a43-453f04b2b295 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:16.640122 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-8abc09ed-443f-48d5-92ff-b29fb0f32a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:16.657850 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31393da5-555f-4753-9a43-453f04b2b295 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:16.659674 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31393da5-555f-4753-9a43-453f04b2b295 service nova] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:16.661697 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abc09ed-443f-48d5-92ff-b29fb0f32a73 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:16.668807 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70a38bb-2de1-4f1b-b97e-c28a12a74dcb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab599d98b0624a2e89090312bcab713f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:16.669626 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a38bb-2de1-4f1b-b97e-c28a12a74dcb service cinder] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:16.670030 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a38bb-2de1-4f1b-b97e-c28a12a74dcb service cinder] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:16.670557 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a38bb-2de1-4f1b-b97e-c28a12a74dcb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:16.673471 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-f70a38bb-2de1-4f1b-b97e-c28a12a74dcb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:16.675229 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 769/1571] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:08:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:08:16.763576 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70a38bb-2de1-4f1b-b97e-c28a12a74dcb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:16.765132 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70a38bb-2de1-4f1b-b97e-c28a12a74dcb service cinder] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:16.774979 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 803/1572] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:08:16 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 03:08:16.821284 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31393da5-555f-4753-9a43-453f04b2b295 service nova] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:16.822932 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-31393da5-555f-4753-9a43-453f04b2b295 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:16.841678 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31393da5-555f-4753-9a43-453f04b2b295 service nova] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:16.848105 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab877f7-50d6-4515-a014-d26f25a8fbfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:16.849148 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab877f7-50d6-4515-a014-d26f25a8fbfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:16.849478 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab877f7-50d6-4515-a014-d26f25a8fbfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:16.855684 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab877f7-50d6-4515-a014-d26f25a8fbfa admin admin] PATH_INFO: `/v3/projects/5c47a804d4d9476c868bcbe3b4097442` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:16.855684 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab877f7-50d6-4515-a014-d26f25a8fbfa admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c47a804d4d9476c868bcbe3b4097442)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:16.855684 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab877f7-50d6-4515-a014-d26f25a8fbfa admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:16.969306 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-1ab877f7-50d6-4515-a014-d26f25a8fbfa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c47a804d4d9476c868bcbe3b4097442', 'request_id': 'req-1ab877f7-50d6-4515-a014-d26f25a8fbfa'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:16.975770 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-1ab877f7-50d6-4515-a014-d26f25a8fbfa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c47a804d4d9476c868bcbe3b4097442', 'request_id': 'req-1ab877f7-50d6-4515-a014-d26f25a8fbfa'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:17.039875 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-1ab877f7-50d6-4515-a014-d26f25a8fbfa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c47a804d4d9476c868bcbe3b4097442', 'request_id': 'req-1ab877f7-50d6-4515-a014-d26f25a8fbfa'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:17.055290 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-1ab877f7-50d6-4515-a014-d26f25a8fbfa admin admin] The token cache is being invalidate because project 5c47a804d4d9476c868bcbe3b4097442 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:08:17.055683 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-1ab877f7-50d6-4515-a014-d26f25a8fbfa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:17.056482 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-1ab877f7-50d6-4515-a014-d26f25a8fbfa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:17.058158 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 770/1573] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:08:16 2022] DELETE /identity/v3/projects/5c47a804d4d9476c868bcbe3b4097442 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:08:17.062439 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70a38bb-2de1-4f1b-b97e-c28a12a74dcb service cinder] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:17.073633 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70a38bb-2de1-4f1b-b97e-c28a12a74dcb service cinder] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:17.131339 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93264044-bd03-41a6-bc05-b91e1199fc4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:17.131971 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93264044-bd03-41a6-bc05-b91e1199fc4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:17.132304 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93264044-bd03-41a6-bc05-b91e1199fc4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:17.132581 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93264044-bd03-41a6-bc05-b91e1199fc4a admin admin] PATH_INFO: `/v3/users/f8101d3da60542b9beeaff579a177c69` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:17.134354 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93264044-bd03-41a6-bc05-b91e1199fc4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8101d3da60542b9beeaff579a177c69)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:17.136361 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93264044-bd03-41a6-bc05-b91e1199fc4a admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:17.386832 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-93264044-bd03-41a6-bc05-b91e1199fc4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8101d3da60542b9beeaff579a177c69'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:17.388777 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-93264044-bd03-41a6-bc05-b91e1199fc4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8101d3da60542b9beeaff579a177c69'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:17.499153 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-93264044-bd03-41a6-bc05-b91e1199fc4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8101d3da60542b9beeaff579a177c69'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:17.499153 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-93264044-bd03-41a6-bc05-b91e1199fc4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8101d3da60542b9beeaff579a177c69'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:17.517030 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-93264044-bd03-41a6-bc05-b91e1199fc4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8101d3da60542b9beeaff579a177c69'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:17.558448 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 804/1574] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:08:17 2022] DELETE /identity/v3/users/f8101d3da60542b9beeaff579a177c69 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 03:08:17.740316 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab877f7-50d6-4515-a014-d26f25a8fbfa admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:17.741807 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-1ab877f7-50d6-4515-a014-d26f25a8fbfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:17.755875 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab877f7-50d6-4515-a014-d26f25a8fbfa admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:17.818714 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24cb287-2eb7-40b7-b700-847d2f32958b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:17.819365 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24cb287-2eb7-40b7-b700-847d2f32958b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:17.819650 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24cb287-2eb7-40b7-b700-847d2f32958b admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:17.819928 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24cb287-2eb7-40b7-b700-847d2f32958b admin admin] PATH_INFO: `/v3/users/338bc94ab85848698c874cbbd73255f1` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:17.821084 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24cb287-2eb7-40b7-b700-847d2f32958b admin admin] RBAC: Authorizing `identity:delete_user(user_id=338bc94ab85848698c874cbbd73255f1)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:17.823050 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24cb287-2eb7-40b7-b700-847d2f32958b admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:18.036415 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-c24cb287-2eb7-40b7-b700-847d2f32958b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '338bc94ab85848698c874cbbd73255f1'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:18.044688 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-c24cb287-2eb7-40b7-b700-847d2f32958b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '338bc94ab85848698c874cbbd73255f1'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:18.056889 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-c24cb287-2eb7-40b7-b700-847d2f32958b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '338bc94ab85848698c874cbbd73255f1'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:18.057868 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-c24cb287-2eb7-40b7-b700-847d2f32958b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '338bc94ab85848698c874cbbd73255f1'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:18.073903 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93264044-bd03-41a6-bc05-b91e1199fc4a admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:18.073903 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-93264044-bd03-41a6-bc05-b91e1199fc4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:18.089633 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93264044-bd03-41a6-bc05-b91e1199fc4a admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:18.096691 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e105a4-4452-4f26-8594-a1ae5df0d0de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:18.097366 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e105a4-4452-4f26-8594-a1ae5df0d0de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:18.097651 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e105a4-4452-4f26-8594-a1ae5df0d0de admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:18.098018 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e105a4-4452-4f26-8594-a1ae5df0d0de admin admin] PATH_INFO: `/v3/projects/b808bf2e6e374292a020f2394a066af9` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:18.099113 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e105a4-4452-4f26-8594-a1ae5df0d0de admin admin] RBAC: Authorizing `identity:delete_project(project_id=b808bf2e6e374292a020f2394a066af9)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:18.101184 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e105a4-4452-4f26-8594-a1ae5df0d0de admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:18.147696 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-c24cb287-2eb7-40b7-b700-847d2f32958b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '338bc94ab85848698c874cbbd73255f1'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:18.155738 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 771/1575] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:08:17 2022] DELETE /identity/v3/users/338bc94ab85848698c874cbbd73255f1 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:08:18.160296 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c814c535-a6c5-4dcd-ad89-073a9bccacb9 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:18.160807 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c814c535-a6c5-4dcd-ad89-073a9bccacb9 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:18.161915 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c814c535-a6c5-4dcd-ad89-073a9bccacb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:18.181049 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-c814c535-a6c5-4dcd-ad89-073a9bccacb9 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:08:18.182413 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-c814c535-a6c5-4dcd-ad89-073a9bccacb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:18.215320 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-f3e105a4-4452-4f26-8594-a1ae5df0d0de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b808bf2e6e374292a020f2394a066af9', 'request_id': 'req-f3e105a4-4452-4f26-8594-a1ae5df0d0de'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:18.222190 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-f3e105a4-4452-4f26-8594-a1ae5df0d0de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b808bf2e6e374292a020f2394a066af9', 'request_id': 'req-f3e105a4-4452-4f26-8594-a1ae5df0d0de'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:18.252931 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 772/1576] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:08:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:08:18.263803 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c814c535-a6c5-4dcd-ad89-073a9bccacb9 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:18.265193 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-c814c535-a6c5-4dcd-ad89-073a9bccacb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:18.319184 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-f3e105a4-4452-4f26-8594-a1ae5df0d0de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b808bf2e6e374292a020f2394a066af9', 'request_id': 'req-f3e105a4-4452-4f26-8594-a1ae5df0d0de'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:18.328142 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c814c535-a6c5-4dcd-ad89-073a9bccacb9 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:18.331301 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-f3e105a4-4452-4f26-8594-a1ae5df0d0de admin admin] The token cache is being invalidate because project b808bf2e6e374292a020f2394a066af9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81517) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:08:18.331597 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-f3e105a4-4452-4f26-8594-a1ae5df0d0de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:18.332274 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-f3e105a4-4452-4f26-8594-a1ae5df0d0de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:18.333885 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 805/1577] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:08:18 2022] DELETE /identity/v3/projects/b808bf2e6e374292a020f2394a066af9 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 03:08:18.339424 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d66b039-0c5d-4b90-8dc2-fa4de8ea6146 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:18.339714 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d66b039-0c5d-4b90-8dc2-fa4de8ea6146 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:18.340016 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d66b039-0c5d-4b90-8dc2-fa4de8ea6146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:18.359714 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-9d66b039-0c5d-4b90-8dc2-fa4de8ea6146 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:08:18.360857 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-9d66b039-0c5d-4b90-8dc2-fa4de8ea6146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:18.377069 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88aee6e-68ab-474b-a8c4-5e8ec134dab0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:18.377684 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88aee6e-68ab-474b-a8c4-5e8ec134dab0 service neutron] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:18.377996 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88aee6e-68ab-474b-a8c4-5e8ec134dab0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:18.378270 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88aee6e-68ab-474b-a8c4-5e8ec134dab0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:18.380001 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-b88aee6e-68ab-474b-a8c4-5e8ec134dab0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:18.420754 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 806/1578] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:08:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 03:08:18.428159 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1cd679-81ae-412d-96a4-0cb8e7c64c78 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:18.428454 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1cd679-81ae-412d-96a4-0cb8e7c64c78 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:18.428734 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1cd679-81ae-412d-96a4-0cb8e7c64c78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:18.440119 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:18.442948 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88aee6e-68ab-474b-a8c4-5e8ec134dab0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:18.444427 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88aee6e-68ab-474b-a8c4-5e8ec134dab0 service neutron] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:18.452353 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-7a1cd679-81ae-412d-96a4-0cb8e7c64c78 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:08:18.453579 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-7a1cd679-81ae-412d-96a4-0cb8e7c64c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:18.454508 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 773/1579] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:08:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 03:08:18.462263 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 807/1580] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:08:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 03:08:18.466473 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88aee6e-68ab-474b-a8c4-5e8ec134dab0 service neutron] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:18.475840 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88aee6e-68ab-474b-a8c4-5e8ec134dab0 service neutron] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:18.480514 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a336cd68-f8c5-4734-9821-6b67294cee29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:18.481109 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a336cd68-f8c5-4734-9821-6b67294cee29 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:18.485800 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a336cd68-f8c5-4734-9821-6b67294cee29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:18.485800 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a336cd68-f8c5-4734-9821-6b67294cee29 admin admin] PATH_INFO: `/v3/domains` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:18.485800 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a336cd68-f8c5-4734-9821-6b67294cee29 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 03:08:18.485800 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a336cd68-f8c5-4734-9821-6b67294cee29 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:18.485800 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a336cd68-f8c5-4734-9821-6b67294cee29 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:18.497299 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 774/1581] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Oct 14 03:08:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 03:08:18.501922 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1cd679-81ae-412d-96a4-0cb8e7c64c78 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:18.513501 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1cd679-81ae-412d-96a4-0cb8e7c64c78 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:18.519020 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a610377-2e98-4de9-ba09-7684db5a59a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:18.519705 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a610377-2e98-4de9-ba09-7684db5a59a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:18.519999 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a610377-2e98-4de9-ba09-7684db5a59a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:18.520294 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a610377-2e98-4de9-ba09-7684db5a59a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:18.520891 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a610377-2e98-4de9-ba09-7684db5a59a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:18.523018 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a610377-2e98-4de9-ba09-7684db5a59a1 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:18.582667 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:INSERT=2,keystone:DELETE=8,keystone:SELECT=90 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:18.637784 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 808/1582] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:08:18 2022] POST /identity/v3/projects => generated 385 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:08:18.638420 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a336cd68-f8c5-4734-9821-6b67294cee29 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:18.651491 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a336cd68-f8c5-4734-9821-6b67294cee29 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:18.783933 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb511ef2-45c8-45ad-9813-8f7968d3a9cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:18.784670 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb511ef2-45c8-45ad-9813-8f7968d3a9cf admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:18.785052 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb511ef2-45c8-45ad-9813-8f7968d3a9cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:18.785377 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb511ef2-45c8-45ad-9813-8f7968d3a9cf admin admin] PATH_INFO: `/v3/users` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:18.786152 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb511ef2-45c8-45ad-9813-8f7968d3a9cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:18.788191 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb511ef2-45c8-45ad-9813-8f7968d3a9cf admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:18.898906 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 775/1583] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:08:18 2022] POST /identity/v3/users => generated 293 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:08:18.902732 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a610377-2e98-4de9-ba09-7684db5a59a1 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:18.913437 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a610377-2e98-4de9-ba09-7684db5a59a1 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:18.918629 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2072bdba-12dc-4e4f-9832-4fd47da239e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:18.919236 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2072bdba-12dc-4e4f-9832-4fd47da239e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:18.919516 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2072bdba-12dc-4e4f-9832-4fd47da239e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:18.919791 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2072bdba-12dc-4e4f-9832-4fd47da239e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:18.922627 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2072bdba-12dc-4e4f-9832-4fd47da239e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:18.922627 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2072bdba-12dc-4e4f-9832-4fd47da239e7 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:18.933725 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 809/1584] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:08:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:08:18.938111 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb511ef2-45c8-45ad-9813-8f7968d3a9cf admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:18.951595 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb511ef2-45c8-45ad-9813-8f7968d3a9cf admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:18.953075 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a29afa-f9fa-4014-ad80-8115b37f07c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:18.953809 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a29afa-f9fa-4014-ad80-8115b37f07c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:18.954096 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a29afa-f9fa-4014-ad80-8115b37f07c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:18.954371 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a29afa-f9fa-4014-ad80-8115b37f07c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:18.955011 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a29afa-f9fa-4014-ad80-8115b37f07c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:18.956590 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a29afa-f9fa-4014-ad80-8115b37f07c5 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:18.967632 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 776/1585] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:08:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:08:18.972098 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2072bdba-12dc-4e4f-9832-4fd47da239e7 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:18.978532 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a29afa-f9fa-4014-ad80-8115b37f07c5 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:18.980339 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-f8a29afa-f9fa-4014-ad80-8115b37f07c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:18.986560 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2072bdba-12dc-4e4f-9832-4fd47da239e7 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:18.988349 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9602db8-aa97-469b-9594-026452ce913b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:18.989074 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9602db8-aa97-469b-9594-026452ce913b admin admin] REQUEST_METHOD: `PUT` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:18.989360 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9602db8-aa97-469b-9594-026452ce913b admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:18.989634 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9602db8-aa97-469b-9594-026452ce913b admin admin] PATH_INFO: `/v3/projects/df8d43e298c5449f8f0e8c3db7c07a41/users/eca194f38abd4097b5e614983476820c/roles/9bd0470b544240209a420f640c79c06c` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:19.019426 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a29afa-f9fa-4014-ad80-8115b37f07c5 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:19.029546 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9602db8-aa97-469b-9594-026452ce913b admin admin] RBAC: Authorizing `identity:create_grant(project_id=df8d43e298c5449f8f0e8c3db7c07a41, user_id=eca194f38abd4097b5e614983476820c, role_id=9bd0470b544240209a420f640c79c06c)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:19.039596 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9602db8-aa97-469b-9594-026452ce913b admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:19.040465 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03403716-3df4-4244-b0ee-87c7b8c775fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:19.041914 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03403716-3df4-4244-b0ee-87c7b8c775fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:19.041914 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03403716-3df4-4244-b0ee-87c7b8c775fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:19.043452 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03403716-3df4-4244-b0ee-87c7b8c775fa admin admin] PATH_INFO: `/v3/projects/2f5763faebd649fe92aa51027a30a27d` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:19.049784 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03403716-3df4-4244-b0ee-87c7b8c775fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f5763faebd649fe92aa51027a30a27d)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:19.053801 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03403716-3df4-4244-b0ee-87c7b8c775fa admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:19.114385 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 810/1586] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:08:18 2022] PUT /identity/v3/projects/df8d43e298c5449f8f0e8c3db7c07a41/users/eca194f38abd4097b5e614983476820c/roles/9bd0470b544240209a420f640c79c06c => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:08:19.126776 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f583ed-bf12-484b-b0cf-4c3172eeaa76 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:19.127164 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f583ed-bf12-484b-b0cf-4c3172eeaa76 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:19.128747 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f583ed-bf12-484b-b0cf-4c3172eeaa76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:19.162102 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-41f583ed-bf12-484b-b0cf-4c3172eeaa76 None None] MFA Rules not processed for user `eca194f38abd4097b5e614983476820c`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:08:19.163511 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-41f583ed-bf12-484b-b0cf-4c3172eeaa76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:19.197786 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-03403716-3df4-4244-b0ee-87c7b8c775fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f5763faebd649fe92aa51027a30a27d', 'request_id': 'req-03403716-3df4-4244-b0ee-87c7b8c775fa'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:19.207437 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-03403716-3df4-4244-b0ee-87c7b8c775fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f5763faebd649fe92aa51027a30a27d', 'request_id': 'req-03403716-3df4-4244-b0ee-87c7b8c775fa'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:19.236121 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 811/1587] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:08:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:08:19.240373 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f583ed-bf12-484b-b0cf-4c3172eeaa76 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:19.296775 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-03403716-3df4-4244-b0ee-87c7b8c775fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f5763faebd649fe92aa51027a30a27d', 'request_id': 'req-03403716-3df4-4244-b0ee-87c7b8c775fa'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:19.322084 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-03403716-3df4-4244-b0ee-87c7b8c775fa admin admin] The token cache is being invalidate because project 2f5763faebd649fe92aa51027a30a27d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:08:19.322516 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-03403716-3df4-4244-b0ee-87c7b8c775fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:19.324315 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-03403716-3df4-4244-b0ee-87c7b8c775fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:19.331213 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 777/1588] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:08:18 2022] DELETE /identity/v3/projects/2f5763faebd649fe92aa51027a30a27d => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 03:08:19.334152 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03403716-3df4-4244-b0ee-87c7b8c775fa admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:19.337513 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-03403716-3df4-4244-b0ee-87c7b8c775fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:19.369895 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f583ed-bf12-484b-b0cf-4c3172eeaa76 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:19.481393 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03403716-3df4-4244-b0ee-87c7b8c775fa admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:19.489041 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f389d84-ae13-4d1c-9f9e-b2ae40193e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:19.490093 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f389d84-ae13-4d1c-9f9e-b2ae40193e3e admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:19.490463 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f389d84-ae13-4d1c-9f9e-b2ae40193e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:19.495736 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f389d84-ae13-4d1c-9f9e-b2ae40193e3e admin admin] PATH_INFO: `/v3/projects` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:19.495736 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f389d84-ae13-4d1c-9f9e-b2ae40193e3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:19.495736 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f389d84-ae13-4d1c-9f9e-b2ae40193e3e admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:19.500574 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d967340d-f09c-42fe-86ac-980e88d59e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:19.501348 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d967340d-f09c-42fe-86ac-980e88d59e74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:19.501635 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d967340d-f09c-42fe-86ac-980e88d59e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:19.501941 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d967340d-f09c-42fe-86ac-980e88d59e74 admin admin] PATH_INFO: `/v3/users/7add678281f046b8b5c7c58bb90fbb55` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:19.504522 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d967340d-f09c-42fe-86ac-980e88d59e74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7add678281f046b8b5c7c58bb90fbb55)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:19.510653 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d967340d-f09c-42fe-86ac-980e88d59e74 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:19.650790 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 812/1589] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:08:19 2022] POST /identity/v3/projects => generated 387 bytes in 407 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 03:08:19.652016 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f389d84-ae13-4d1c-9f9e-b2ae40193e3e admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:19.654330 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-4f389d84-ae13-4d1c-9f9e-b2ae40193e3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:19.755523 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f389d84-ae13-4d1c-9f9e-b2ae40193e3e admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:19.773863 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ebceb6-5cc5-4bfc-b275-15143b544359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:19.773863 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ebceb6-5cc5-4bfc-b275-15143b544359 admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:19.773863 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ebceb6-5cc5-4bfc-b275-15143b544359 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:19.773863 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ebceb6-5cc5-4bfc-b275-15143b544359 admin admin] PATH_INFO: `/v3/users` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:19.774452 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ebceb6-5cc5-4bfc-b275-15143b544359 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:19.774452 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ebceb6-5cc5-4bfc-b275-15143b544359 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:19.794245 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:19.799087 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-d967340d-f09c-42fe-86ac-980e88d59e74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7add678281f046b8b5c7c58bb90fbb55'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:19.807611 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-d967340d-f09c-42fe-86ac-980e88d59e74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7add678281f046b8b5c7c58bb90fbb55'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:19.824637 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-d967340d-f09c-42fe-86ac-980e88d59e74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7add678281f046b8b5c7c58bb90fbb55'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:19.825523 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-d967340d-f09c-42fe-86ac-980e88d59e74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7add678281f046b8b5c7c58bb90fbb55'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:19.850947 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 813/1590] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:08:19 2022] POST /identity/v3/users => generated 294 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:08:19.855086 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ebceb6-5cc5-4bfc-b275-15143b544359 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:19.857458 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-92ebceb6-5cc5-4bfc-b275-15143b544359 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:19.871420 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ebceb6-5cc5-4bfc-b275-15143b544359 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:19.876254 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac9fc40-4cff-4506-9e08-a717eea4b3ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:19.876901 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac9fc40-4cff-4506-9e08-a717eea4b3ce admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:19.877248 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac9fc40-4cff-4506-9e08-a717eea4b3ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:19.877706 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac9fc40-4cff-4506-9e08-a717eea4b3ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:19.878590 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac9fc40-4cff-4506-9e08-a717eea4b3ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:19.880286 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac9fc40-4cff-4506-9e08-a717eea4b3ce admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:19.893034 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 814/1591] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:08:19 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:08:19.896600 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac9fc40-4cff-4506-9e08-a717eea4b3ce admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:19.910703 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac9fc40-4cff-4506-9e08-a717eea4b3ce admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:19.914435 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-d967340d-f09c-42fe-86ac-980e88d59e74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7add678281f046b8b5c7c58bb90fbb55'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:19.924237 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 778/1592] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:08:19 2022] DELETE /identity/v3/users/7add678281f046b8b5c7c58bb90fbb55 => generated 0 bytes in 592 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:08:19.924903 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c942d0-6cbf-403f-9c43-96802d25b484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:19.925642 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c942d0-6cbf-403f-9c43-96802d25b484 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:19.926621 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c942d0-6cbf-403f-9c43-96802d25b484 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:19.926912 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c942d0-6cbf-403f-9c43-96802d25b484 admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:19.927584 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c942d0-6cbf-403f-9c43-96802d25b484 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:19.929227 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c942d0-6cbf-403f-9c43-96802d25b484 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:19.946198 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 815/1593] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:08:19 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 03:08:19.948442 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d967340d-f09c-42fe-86ac-980e88d59e74 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:20.068488 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d967340d-f09c-42fe-86ac-980e88d59e74 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:20.076218 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2691debd-b2f7-4974-9740-b7642458d674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:20.076906 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2691debd-b2f7-4974-9740-b7642458d674 admin admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:20.077178 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2691debd-b2f7-4974-9740-b7642458d674 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:20.077460 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2691debd-b2f7-4974-9740-b7642458d674 admin admin] PATH_INFO: `/v3/projects/56b6a1a123da472982f7a66a900015f1/users/d06f9702a94e45cb88ddfceffadcdeda/roles/9bd0470b544240209a420f640c79c06c` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:20.098034 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2691debd-b2f7-4974-9740-b7642458d674 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56b6a1a123da472982f7a66a900015f1, user_id=d06f9702a94e45cb88ddfceffadcdeda, role_id=9bd0470b544240209a420f640c79c06c)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:20.101997 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2691debd-b2f7-4974-9740-b7642458d674 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:20.176906 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 779/1594] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:08:19 2022] PUT /identity/v3/projects/56b6a1a123da472982f7a66a900015f1/users/d06f9702a94e45cb88ddfceffadcdeda/roles/9bd0470b544240209a420f640c79c06c => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 03:08:20.188077 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb61c3e0-f530-4035-bd36-9bc0fb1ba108 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:20.193585 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb61c3e0-f530-4035-bd36-9bc0fb1ba108 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:20.193585 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb61c3e0-f530-4035-bd36-9bc0fb1ba108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:20.220244 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-bb61c3e0-f530-4035-bd36-9bc0fb1ba108 None None] MFA Rules not processed for user `d06f9702a94e45cb88ddfceffadcdeda`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:08:20.221435 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-bb61c3e0-f530-4035-bd36-9bc0fb1ba108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:20.308491 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 816/1595] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:08:20 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:08:20.315136 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2691debd-b2f7-4974-9740-b7642458d674 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:20.385568 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2691debd-b2f7-4974-9740-b7642458d674 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:20.392065 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f73c8ed-0774-4bfe-8e43-796a4431ad9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:20.392693 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f73c8ed-0774-4bfe-8e43-796a4431ad9c admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:20.393010 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f73c8ed-0774-4bfe-8e43-796a4431ad9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:20.393365 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f73c8ed-0774-4bfe-8e43-796a4431ad9c admin admin] PATH_INFO: `/v3/projects` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:20.397403 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f73c8ed-0774-4bfe-8e43-796a4431ad9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:20.397403 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f73c8ed-0774-4bfe-8e43-796a4431ad9c admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:20.540673 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb61c3e0-f530-4035-bd36-9bc0fb1ba108 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:20.543272 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-bb61c3e0-f530-4035-bd36-9bc0fb1ba108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:20.560802 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 780/1596] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:08:20 2022] POST /identity/v3/projects => generated 387 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:08:20.561100 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb61c3e0-f530-4035-bd36-9bc0fb1ba108 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:20.565023 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f73c8ed-0774-4bfe-8e43-796a4431ad9c admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:20.626165 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7342d870-4c43-47fe-afa9-4fa208c03b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:20.626977 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7342d870-4c43-47fe-afa9-4fa208c03b0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:20.627264 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7342d870-4c43-47fe-afa9-4fa208c03b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:20.627538 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7342d870-4c43-47fe-afa9-4fa208c03b0c admin admin] PATH_INFO: `/v3/projects/d33a6ff5d3e3467eb0b4d82a5bde15ce` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:20.628654 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7342d870-4c43-47fe-afa9-4fa208c03b0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d33a6ff5d3e3467eb0b4d82a5bde15ce)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:20.630671 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7342d870-4c43-47fe-afa9-4fa208c03b0c admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:20.642732 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f73c8ed-0774-4bfe-8e43-796a4431ad9c admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:20.647266 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fd3343-c55b-4a6f-ba50-2c663602a445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:20.647872 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fd3343-c55b-4a6f-ba50-2c663602a445 admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:20.648165 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fd3343-c55b-4a6f-ba50-2c663602a445 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:20.648442 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fd3343-c55b-4a6f-ba50-2c663602a445 admin admin] PATH_INFO: `/v3/users` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:20.649076 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fd3343-c55b-4a6f-ba50-2c663602a445 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:20.651049 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fd3343-c55b-4a6f-ba50-2c663602a445 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:20.680149 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:20.759857 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-7342d870-4c43-47fe-afa9-4fa208c03b0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd33a6ff5d3e3467eb0b4d82a5bde15ce', 'request_id': 'req-7342d870-4c43-47fe-afa9-4fa208c03b0c'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:20.766293 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-7342d870-4c43-47fe-afa9-4fa208c03b0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd33a6ff5d3e3467eb0b4d82a5bde15ce', 'request_id': 'req-7342d870-4c43-47fe-afa9-4fa208c03b0c'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:20.783797 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 781/1597] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:08:20 2022] POST /identity/v3/users => generated 300 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 03:08:20.784708 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fd3343-c55b-4a6f-ba50-2c663602a445 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:20.796369 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fd3343-c55b-4a6f-ba50-2c663602a445 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:20.852717 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-7342d870-4c43-47fe-afa9-4fa208c03b0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd33a6ff5d3e3467eb0b4d82a5bde15ce', 'request_id': 'req-7342d870-4c43-47fe-afa9-4fa208c03b0c'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:20.861663 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0f9b9c-7a17-45df-9caf-c1ecf859accf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:20.862500 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0f9b9c-7a17-45df-9caf-c1ecf859accf admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:20.862856 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0f9b9c-7a17-45df-9caf-c1ecf859accf admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:20.863029 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0f9b9c-7a17-45df-9caf-c1ecf859accf admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:20.863729 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0f9b9c-7a17-45df-9caf-c1ecf859accf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:20.865948 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0f9b9c-7a17-45df-9caf-c1ecf859accf admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:20.877155 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-7342d870-4c43-47fe-afa9-4fa208c03b0c admin admin] The token cache is being invalidate because project d33a6ff5d3e3467eb0b4d82a5bde15ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81517) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:08:20.877729 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-7342d870-4c43-47fe-afa9-4fa208c03b0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:20.878616 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-7342d870-4c43-47fe-afa9-4fa208c03b0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:20.881463 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 782/1598] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:08:20 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:08:20.881463 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:20.881637 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 817/1599] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:08:20 2022] DELETE /identity/v3/projects/d33a6ff5d3e3467eb0b4d82a5bde15ce => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:08:20.883192 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0f9b9c-7a17-45df-9caf-c1ecf859accf admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:20.885260 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-5a0f9b9c-7a17-45df-9caf-c1ecf859accf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:20.885715 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710c979c-1480-4bec-89d2-cbb96908ee71 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:20.886622 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710c979c-1480-4bec-89d2-cbb96908ee71 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:20.886958 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710c979c-1480-4bec-89d2-cbb96908ee71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:20.898821 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0f9b9c-7a17-45df-9caf-c1ecf859accf admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:20.903906 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126041a1-a3de-4a81-aec1-e72d620f6bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:20.904646 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126041a1-a3de-4a81-aec1-e72d620f6bed admin admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:20.905020 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126041a1-a3de-4a81-aec1-e72d620f6bed admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:20.905298 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126041a1-a3de-4a81-aec1-e72d620f6bed admin admin] PATH_INFO: `/v3/projects/ac9ea53b915248fa92bc5dfb5fc85785/users/62f14f2eb3c9491cb86fe6142abb18e5/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:20.911802 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-710c979c-1480-4bec-89d2-cbb96908ee71 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:08:20.913209 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-710c979c-1480-4bec-89d2-cbb96908ee71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:20.923446 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 818/1600] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:08:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 03:08:20.924140 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126041a1-a3de-4a81-aec1-e72d620f6bed admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac9ea53b915248fa92bc5dfb5fc85785, user_id=62f14f2eb3c9491cb86fe6142abb18e5, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:20.927032 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126041a1-a3de-4a81-aec1-e72d620f6bed admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:20.930830 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce1e97-2728-43a4-8c9e-92f6828c476d None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:20.931295 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce1e97-2728-43a4-8c9e-92f6828c476d None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:20.931632 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce1e97-2728-43a4-8c9e-92f6828c476d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:20.951649 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-ebce1e97-2728-43a4-8c9e-92f6828c476d None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:08:20.952859 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-ebce1e97-2728-43a4-8c9e-92f6828c476d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:20.961808 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 819/1601] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:08:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:08:20.965914 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebce1e97-2728-43a4-8c9e-92f6828c476d None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:20.975174 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebce1e97-2728-43a4-8c9e-92f6828c476d None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:20.980647 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7026e345-b276-414b-b9ec-33463756d8e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:20.981244 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7026e345-b276-414b-b9ec-33463756d8e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:20.981527 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7026e345-b276-414b-b9ec-33463756d8e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:20.981834 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7026e345-b276-414b-b9ec-33463756d8e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:20.982485 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7026e345-b276-414b-b9ec-33463756d8e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 03:08:20.982792 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7026e345-b276-414b-b9ec-33463756d8e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:20.984955 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7026e345-b276-414b-b9ec-33463756d8e4 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:20.994173 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 783/1602] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:08:20 2022] PUT /identity/v3/projects/ac9ea53b915248fa92bc5dfb5fc85785/users/62f14f2eb3c9491cb86fe6142abb18e5/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:08:20.997230 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 820/1603] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Oct 14 03:08:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 03:08:21.000356 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126041a1-a3de-4a81-aec1-e72d620f6bed admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:21.001516 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7026e345-b276-414b-b9ec-33463756d8e4 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:21.014338 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7026e345-b276-414b-b9ec-33463756d8e4 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:21.078371 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164b8082-68d4-43a9-8bf6-60c27ee15977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:21.078977 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164b8082-68d4-43a9-8bf6-60c27ee15977 admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:21.079256 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164b8082-68d4-43a9-8bf6-60c27ee15977 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:21.079529 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164b8082-68d4-43a9-8bf6-60c27ee15977 admin admin] PATH_INFO: `/v3/projects` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:21.080156 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164b8082-68d4-43a9-8bf6-60c27ee15977 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:21.082143 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164b8082-68d4-43a9-8bf6-60c27ee15977 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:21.104051 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126041a1-a3de-4a81-aec1-e72d620f6bed admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:21.109259 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac61583-c5c2-449d-979b-1cd94cf28fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:21.109908 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac61583-c5c2-449d-979b-1cd94cf28fc6 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:21.110196 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac61583-c5c2-449d-979b-1cd94cf28fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:21.110475 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac61583-c5c2-449d-979b-1cd94cf28fc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:21.111129 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac61583-c5c2-449d-979b-1cd94cf28fc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:21.112728 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac61583-c5c2-449d-979b-1cd94cf28fc6 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:21.123627 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 784/1604] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:08:20 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:08:21.127730 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac61583-c5c2-449d-979b-1cd94cf28fc6 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:21.138526 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac61583-c5c2-449d-979b-1cd94cf28fc6 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:21.143727 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a2b77b-bbe7-439c-ad6f-13dc4714828f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:21.144407 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a2b77b-bbe7-439c-ad6f-13dc4714828f admin admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:21.144694 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a2b77b-bbe7-439c-ad6f-13dc4714828f admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:21.144972 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a2b77b-bbe7-439c-ad6f-13dc4714828f admin admin] PATH_INFO: `/v3/projects/ac9ea53b915248fa92bc5dfb5fc85785/users/62f14f2eb3c9491cb86fe6142abb18e5/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:21.147500 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a2b77b-bbe7-439c-ad6f-13dc4714828f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac9ea53b915248fa92bc5dfb5fc85785, user_id=62f14f2eb3c9491cb86fe6142abb18e5, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:21.151712 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a2b77b-bbe7-439c-ad6f-13dc4714828f admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:21.161842 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 785/1605] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:08:21 2022] PUT /identity/v3/projects/ac9ea53b915248fa92bc5dfb5fc85785/users/62f14f2eb3c9491cb86fe6142abb18e5/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 03:08:21.172233 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f34f10-807b-47d4-8214-5f9756b91872 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:21.172527 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f34f10-807b-47d4-8214-5f9756b91872 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:21.172811 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f34f10-807b-47d4-8214-5f9756b91872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:21.185811 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 821/1606] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:08:21 2022] POST /identity/v3/projects => generated 379 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:08:21.187742 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164b8082-68d4-43a9-8bf6-60c27ee15977 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:21.216167 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-d7f34f10-807b-47d4-8214-5f9756b91872 None None] MFA Rules not processed for user `62f14f2eb3c9491cb86fe6142abb18e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:08:21.217648 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-d7f34f10-807b-47d4-8214-5f9756b91872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:21.263577 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164b8082-68d4-43a9-8bf6-60c27ee15977 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:21.272632 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123d6f39-2472-42c6-8b86-c5cda53e9512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:21.274434 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123d6f39-2472-42c6-8b86-c5cda53e9512 admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:21.274721 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123d6f39-2472-42c6-8b86-c5cda53e9512 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:21.275140 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123d6f39-2472-42c6-8b86-c5cda53e9512 admin admin] PATH_INFO: `/v3/users` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:21.275928 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123d6f39-2472-42c6-8b86-c5cda53e9512 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:21.280382 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123d6f39-2472-42c6-8b86-c5cda53e9512 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:21.310495 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 786/1607] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:08:21 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 03:08:21.321641 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f34f10-807b-47d4-8214-5f9756b91872 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:21.323137 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-d7f34f10-807b-47d4-8214-5f9756b91872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:21.379523 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f34f10-807b-47d4-8214-5f9756b91872 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:21.381797 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 822/1608] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:08:21 2022] POST /identity/v3/users => generated 290 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:08:21.386388 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123d6f39-2472-42c6-8b86-c5cda53e9512 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:21.387995 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46409a5-374b-4e6f-8dd4-93dad9724330 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab599d98b0624a2e89090312bcab713f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:21.389035 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46409a5-374b-4e6f-8dd4-93dad9724330 service cinder] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:21.389448 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46409a5-374b-4e6f-8dd4-93dad9724330 service cinder] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:21.389732 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46409a5-374b-4e6f-8dd4-93dad9724330 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:21.398973 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123d6f39-2472-42c6-8b86-c5cda53e9512 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:21.400680 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46409a5-374b-4e6f-8dd4-93dad9724330 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:21.402487 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46409a5-374b-4e6f-8dd4-93dad9724330 service cinder] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:21.403940 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e265950-b801-4472-9c3a-2f6cddfe596c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:21.404615 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e265950-b801-4472-9c3a-2f6cddfe596c admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:21.404906 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e265950-b801-4472-9c3a-2f6cddfe596c admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:21.405216 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e265950-b801-4472-9c3a-2f6cddfe596c admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:21.405864 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e265950-b801-4472-9c3a-2f6cddfe596c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:21.407491 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e265950-b801-4472-9c3a-2f6cddfe596c admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:21.413016 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 787/1609] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:08:21 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 03:08:21.420802 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 823/1610] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:08:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 03:08:21.422474 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46409a5-374b-4e6f-8dd4-93dad9724330 service cinder] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:21.433346 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46409a5-374b-4e6f-8dd4-93dad9724330 service cinder] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:21.438700 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d56900d-741e-4380-9e84-3dd7d0a77c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:21.444149 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d56900d-741e-4380-9e84-3dd7d0a77c29 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:21.444149 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d56900d-741e-4380-9e84-3dd7d0a77c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:21.444149 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d56900d-741e-4380-9e84-3dd7d0a77c29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:21.444149 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d56900d-741e-4380-9e84-3dd7d0a77c29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:21.444149 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d56900d-741e-4380-9e84-3dd7d0a77c29 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:21.456558 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 788/1611] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:08:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:08:21.457587 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e265950-b801-4472-9c3a-2f6cddfe596c admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:21.468669 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e265950-b801-4472-9c3a-2f6cddfe596c admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:21.474007 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f7c161-1be8-4de8-8ba6-d029a20840c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:21.479234 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f7c161-1be8-4de8-8ba6-d029a20840c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:21.479234 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f7c161-1be8-4de8-8ba6-d029a20840c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:21.479234 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f7c161-1be8-4de8-8ba6-d029a20840c7 admin admin] PATH_INFO: `/v3/projects/00c0c89d0bb743f89a3f3c9ea67ef53e/users/84f5fbc323c24fd88c87f8e7bb083e0a/roles/9bd0470b544240209a420f640c79c06c` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:21.494031 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f7c161-1be8-4de8-8ba6-d029a20840c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00c0c89d0bb743f89a3f3c9ea67ef53e, user_id=84f5fbc323c24fd88c87f8e7bb083e0a, role_id=9bd0470b544240209a420f640c79c06c)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:21.496871 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f7c161-1be8-4de8-8ba6-d029a20840c7 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:21.576993 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 824/1612] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:08:21 2022] PUT /identity/v3/projects/00c0c89d0bb743f89a3f3c9ea67ef53e/users/84f5fbc323c24fd88c87f8e7bb083e0a/roles/9bd0470b544240209a420f640c79c06c => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 03:08:21.588365 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2b9ac7-828a-4371-bb09-042966c2725e None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:21.588655 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2b9ac7-828a-4371-bb09-042966c2725e None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:21.588930 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2b9ac7-828a-4371-bb09-042966c2725e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:21.624039 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-9d2b9ac7-828a-4371-bb09-042966c2725e None None] MFA Rules not processed for user `84f5fbc323c24fd88c87f8e7bb083e0a`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:08:21.624039 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-9d2b9ac7-828a-4371-bb09-042966c2725e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:21.685231 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 789/1613] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:08:21 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:08:21.689862 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f7c161-1be8-4de8-8ba6-d029a20840c7 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:21.753707 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f7c161-1be8-4de8-8ba6-d029a20840c7 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:21.759479 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a4ac9c-bed2-4253-ab6f-9c2b7103a8db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:21.760077 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a4ac9c-bed2-4253-ab6f-9c2b7103a8db admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:21.760356 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a4ac9c-bed2-4253-ab6f-9c2b7103a8db admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:21.760628 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a4ac9c-bed2-4253-ab6f-9c2b7103a8db admin admin] PATH_INFO: `/v3/projects` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:21.761268 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a4ac9c-bed2-4253-ab6f-9c2b7103a8db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:21.763306 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a4ac9c-bed2-4253-ab6f-9c2b7103a8db admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:21.817341 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 825/1614] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:08:21 2022] POST /identity/v3/projects => generated 379 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:08:21.822236 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2b9ac7-828a-4371-bb09-042966c2725e None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:21.832832 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2b9ac7-828a-4371-bb09-042966c2725e None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:21.887104 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd08f5e7-1e9f-427b-837b-c1aa01f756a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:21.887814 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd08f5e7-1e9f-427b-837b-c1aa01f756a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:21.887992 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd08f5e7-1e9f-427b-837b-c1aa01f756a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:21.888268 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd08f5e7-1e9f-427b-837b-c1aa01f756a4 admin admin] PATH_INFO: `/v3/users` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:21.888914 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd08f5e7-1e9f-427b-837b-c1aa01f756a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:21.890924 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd08f5e7-1e9f-427b-837b-c1aa01f756a4 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:21.993557 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:21.998693 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 790/1615] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:08:21 2022] POST /identity/v3/users => generated 290 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:08:22.002842 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a4ac9c-bed2-4253-ab6f-9c2b7103a8db admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:22.013241 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a4ac9c-bed2-4253-ab6f-9c2b7103a8db admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:22.018127 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7f1db7-0dc2-40ad-aa74-f7e642e63c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:22.018739 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7f1db7-0dc2-40ad-aa74-f7e642e63c49 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:22.019022 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7f1db7-0dc2-40ad-aa74-f7e642e63c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:22.019300 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7f1db7-0dc2-40ad-aa74-f7e642e63c49 admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:22.019955 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7f1db7-0dc2-40ad-aa74-f7e642e63c49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:22.021557 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7f1db7-0dc2-40ad-aa74-f7e642e63c49 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:22.032342 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 826/1616] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:08:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:08:22.036253 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd08f5e7-1e9f-427b-837b-c1aa01f756a4 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:22.047118 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd08f5e7-1e9f-427b-837b-c1aa01f756a4 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:22.055709 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4af92e-06a0-4ff8-9ffa-5d8f18d7cdc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:22.055709 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4af92e-06a0-4ff8-9ffa-5d8f18d7cdc0 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:22.055709 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4af92e-06a0-4ff8-9ffa-5d8f18d7cdc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:22.055709 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4af92e-06a0-4ff8-9ffa-5d8f18d7cdc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:22.056091 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4af92e-06a0-4ff8-9ffa-5d8f18d7cdc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:22.057138 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4af92e-06a0-4ff8-9ffa-5d8f18d7cdc0 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:22.070121 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 791/1617] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:08:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:08:22.074983 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7f1db7-0dc2-40ad-aa74-f7e642e63c49 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:22.100764 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7f1db7-0dc2-40ad-aa74-f7e642e63c49 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:22.106001 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e66a15-1d2c-4909-93c5-1274daa50797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:22.106702 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e66a15-1d2c-4909-93c5-1274daa50797 admin admin] REQUEST_METHOD: `PUT` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:22.106991 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e66a15-1d2c-4909-93c5-1274daa50797 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:22.107267 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e66a15-1d2c-4909-93c5-1274daa50797 admin admin] PATH_INFO: `/v3/projects/89fadb3a6cb54d599fddef24a3eb945e/users/9a22abbb2e344596a0c33c26e82fb741/roles/9bd0470b544240209a420f640c79c06c` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:22.124026 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e66a15-1d2c-4909-93c5-1274daa50797 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89fadb3a6cb54d599fddef24a3eb945e, user_id=9a22abbb2e344596a0c33c26e82fb741, role_id=9bd0470b544240209a420f640c79c06c)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:22.126791 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e66a15-1d2c-4909-93c5-1274daa50797 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:22.199725 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 827/1618] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:08:22 2022] PUT /identity/v3/projects/89fadb3a6cb54d599fddef24a3eb945e/users/9a22abbb2e344596a0c33c26e82fb741/roles/9bd0470b544240209a420f640c79c06c => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:08:22.210723 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a014aa6-105a-42c7-877e-1f7a48420303 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:22.212090 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a014aa6-105a-42c7-877e-1f7a48420303 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:22.212374 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a014aa6-105a-42c7-877e-1f7a48420303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:22.230000 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:22.243675 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-2a014aa6-105a-42c7-877e-1f7a48420303 None None] MFA Rules not processed for user `9a22abbb2e344596a0c33c26e82fb741`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:08:22.244889 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-2a014aa6-105a-42c7-877e-1f7a48420303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:22.327383 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 792/1619] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:08:22 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:08:22.331700 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e66a15-1d2c-4909-93c5-1274daa50797 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:22.412251 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e66a15-1d2c-4909-93c5-1274daa50797 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:22.418464 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad79df4-f9fb-4094-8571-fe4efd1b03ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:22.419243 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad79df4-f9fb-4094-8571-fe4efd1b03ae admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:22.419856 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad79df4-f9fb-4094-8571-fe4efd1b03ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:22.420727 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad79df4-f9fb-4094-8571-fe4efd1b03ae admin admin] PATH_INFO: `/v3/projects` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:22.421481 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad79df4-f9fb-4094-8571-fe4efd1b03ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:22.423795 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad79df4-f9fb-4094-8571-fe4efd1b03ae admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:22.576341 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 828/1620] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:08:22 2022] POST /identity/v3/projects => generated 379 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:08:22.580139 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a014aa6-105a-42c7-877e-1f7a48420303 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:22.589311 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad79df4-f9fb-4094-8571-fe4efd1b03ae admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:22.595237 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a014aa6-105a-42c7-877e-1f7a48420303 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:22.597483 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-1ad79df4-f9fb-4094-8571-fe4efd1b03ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:22.714376 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad79df4-f9fb-4094-8571-fe4efd1b03ae admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:22.724727 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9601f92e-c57d-4066-aceb-dcda8b0f93ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab599d98b0624a2e89090312bcab713f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:22.725706 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9601f92e-c57d-4066-aceb-dcda8b0f93ef service cinder] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:22.726165 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9601f92e-c57d-4066-aceb-dcda8b0f93ef service cinder] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:22.727107 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9601f92e-c57d-4066-aceb-dcda8b0f93ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:22.731918 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-9601f92e-c57d-4066-aceb-dcda8b0f93ef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:22.744316 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff5f939-3302-454a-82c4-bbd6b36f39c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:22.745196 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff5f939-3302-454a-82c4-bbd6b36f39c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:22.745622 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff5f939-3302-454a-82c4-bbd6b36f39c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:22.746086 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff5f939-3302-454a-82c4-bbd6b36f39c8 admin admin] PATH_INFO: `/v3/users` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:22.746899 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff5f939-3302-454a-82c4-bbd6b36f39c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:22.749177 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff5f939-3302-454a-82c4-bbd6b36f39c8 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:22.799842 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9601f92e-c57d-4066-aceb-dcda8b0f93ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:22.801426 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9601f92e-c57d-4066-aceb-dcda8b0f93ef service cinder] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:22.810905 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 829/1621] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:08:22 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:08:22.823645 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 793/1622] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:08:22 2022] POST /identity/v3/users => generated 296 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:08:22.828239 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9601f92e-c57d-4066-aceb-dcda8b0f93ef service cinder] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:22.844566 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9601f92e-c57d-4066-aceb-dcda8b0f93ef service cinder] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:22.852138 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfd9c19-9474-4a76-a4c8-45104a602c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:22.858768 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd9c19-9474-4a76-a4c8-45104a602c10 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:22.858768 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd9c19-9474-4a76-a4c8-45104a602c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:22.858768 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd9c19-9474-4a76-a4c8-45104a602c10 admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:22.858768 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfd9c19-9474-4a76-a4c8-45104a602c10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:22.859775 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfd9c19-9474-4a76-a4c8-45104a602c10 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:22.871765 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 830/1623] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:08:22 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:08:22.881959 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff5f939-3302-454a-82c4-bbd6b36f39c8 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:22.915109 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff5f939-3302-454a-82c4-bbd6b36f39c8 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:22.922553 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80dde90-14b7-46da-bd2b-6c894a151692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:22.924450 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80dde90-14b7-46da-bd2b-6c894a151692 admin admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:22.924904 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80dde90-14b7-46da-bd2b-6c894a151692 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:22.925288 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80dde90-14b7-46da-bd2b-6c894a151692 admin admin] PATH_INFO: `/v3/projects/7845ac7ad3bc40b693af52ba81517309/users/8397903bcaa140899202b00cd0e39ec2/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:22.968603 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80dde90-14b7-46da-bd2b-6c894a151692 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7845ac7ad3bc40b693af52ba81517309, user_id=8397903bcaa140899202b00cd0e39ec2, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:22.972249 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80dde90-14b7-46da-bd2b-6c894a151692 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:23.050563 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:23.102536 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 794/1624] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:08:22 2022] PUT /identity/v3/projects/7845ac7ad3bc40b693af52ba81517309/users/8397903bcaa140899202b00cd0e39ec2/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:08:23.106661 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfd9c19-9474-4a76-a4c8-45104a602c10 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:23.118149 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfd9c19-9474-4a76-a4c8-45104a602c10 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:23.177702 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aee011-6ba4-4fe7-89c2-4d858b9a4f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:23.178364 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aee011-6ba4-4fe7-89c2-4d858b9a4f23 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:23.178643 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aee011-6ba4-4fe7-89c2-4d858b9a4f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:23.178910 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aee011-6ba4-4fe7-89c2-4d858b9a4f23 admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:23.180036 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3aee011-6ba4-4fe7-89c2-4d858b9a4f23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:23.181632 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3aee011-6ba4-4fe7-89c2-4d858b9a4f23 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:23.191949 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 831/1625] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:08:23 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:08:23.196756 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80dde90-14b7-46da-bd2b-6c894a151692 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:23.207723 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80dde90-14b7-46da-bd2b-6c894a151692 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:23.213419 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0afd7d-9d7b-41e3-b20b-4997d6b0195e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:23.214121 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0afd7d-9d7b-41e3-b20b-4997d6b0195e admin admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:23.214498 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0afd7d-9d7b-41e3-b20b-4997d6b0195e admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:23.214759 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0afd7d-9d7b-41e3-b20b-4997d6b0195e admin admin] PATH_INFO: `/v3/projects/7845ac7ad3bc40b693af52ba81517309/users/8397903bcaa140899202b00cd0e39ec2/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:23.216771 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0afd7d-9d7b-41e3-b20b-4997d6b0195e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7845ac7ad3bc40b693af52ba81517309, user_id=8397903bcaa140899202b00cd0e39ec2, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:23.219554 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0afd7d-9d7b-41e3-b20b-4997d6b0195e admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:23.230962 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 795/1626] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:08:23 2022] PUT /identity/v3/projects/7845ac7ad3bc40b693af52ba81517309/users/8397903bcaa140899202b00cd0e39ec2/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 03:08:23.241771 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e922bf71-4904-40d8-8df1-3a8e7e6633b7 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:23.242580 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e922bf71-4904-40d8-8df1-3a8e7e6633b7 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:23.242862 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e922bf71-4904-40d8-8df1-3a8e7e6633b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:23.272396 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-e922bf71-4904-40d8-8df1-3a8e7e6633b7 None None] MFA Rules not processed for user `8397903bcaa140899202b00cd0e39ec2`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:08:23.273636 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-e922bf71-4904-40d8-8df1-3a8e7e6633b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:23.389595 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:23.413155 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 832/1627] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:08:23 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:08:23.546582 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0afd7d-9d7b-41e3-b20b-4997d6b0195e admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:23.653311 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0afd7d-9d7b-41e3-b20b-4997d6b0195e admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:23.669365 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e99bfa-803d-4065-94e9-d7daa1d0bf5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab599d98b0624a2e89090312bcab713f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:23.669693 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e99bfa-803d-4065-94e9-d7daa1d0bf5e service cinder] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:23.670005 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e99bfa-803d-4065-94e9-d7daa1d0bf5e service cinder] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:23.670282 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e99bfa-803d-4065-94e9-d7daa1d0bf5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:23.681590 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e99bfa-803d-4065-94e9-d7daa1d0bf5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:23.683409 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e99bfa-803d-4065-94e9-d7daa1d0bf5e service cinder] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:23.695662 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 796/1628] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:08:23 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:08:23.860624 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e922bf71-4904-40d8-8df1-3a8e7e6633b7 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:23.870417 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e922bf71-4904-40d8-8df1-3a8e7e6633b7 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:23.874053 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56510b5-88df-42df-b477-145a0c0d32e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab599d98b0624a2e89090312bcab713f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:23.878621 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56510b5-88df-42df-b477-145a0c0d32e7 service cinder] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:23.878621 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56510b5-88df-42df-b477-145a0c0d32e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:23.878621 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56510b5-88df-42df-b477-145a0c0d32e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:23.937743 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56510b5-88df-42df-b477-145a0c0d32e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:23.940317 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56510b5-88df-42df-b477-145a0c0d32e7 service cinder] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:23.950331 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 833/1629] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:08:23 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:08:24.136909 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:INSERT=3,keystone:SELECT=92,keystone:DELETE=5 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:24.404998 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:25.181340 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:25.257095 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:DELETE=2,keystone:SELECT=93,keystone:INSERT=5 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:25.486590 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:25.730954 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:25.816584 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:26.793570 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:26.833386 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e99bfa-803d-4065-94e9-d7daa1d0bf5e service cinder] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:26.834765 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-b4e99bfa-803d-4065-94e9-d7daa1d0bf5e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:27.027990 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e99bfa-803d-4065-94e9-d7daa1d0bf5e service cinder] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:27.032641 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c017055-082c-462a-b23e-e4152db526ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:27.033273 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c017055-082c-462a-b23e-e4152db526ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:27.033555 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c017055-082c-462a-b23e-e4152db526ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:27.033865 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c017055-082c-462a-b23e-e4152db526ef admin admin] PATH_INFO: `/v3/users/cd1001992c314b6baf2d00552eb930a8` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:27.034988 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c017055-082c-462a-b23e-e4152db526ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd1001992c314b6baf2d00552eb930a8)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:27.036906 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c017055-082c-462a-b23e-e4152db526ef admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:27.231067 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-8c017055-082c-462a-b23e-e4152db526ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd1001992c314b6baf2d00552eb930a8'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:27.240660 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-8c017055-082c-462a-b23e-e4152db526ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd1001992c314b6baf2d00552eb930a8'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:27.255091 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-8c017055-082c-462a-b23e-e4152db526ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd1001992c314b6baf2d00552eb930a8'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:27.255765 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-8c017055-082c-462a-b23e-e4152db526ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd1001992c314b6baf2d00552eb930a8'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:27.364614 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-8c017055-082c-462a-b23e-e4152db526ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd1001992c314b6baf2d00552eb930a8'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:27.381742 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 797/1630] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:08:26 2022] DELETE /identity/v3/users/cd1001992c314b6baf2d00552eb930a8 => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:08:27.382299 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b74c8c-ade7-4e6e-88be-45b368ae03e7 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:27.382626 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b74c8c-ade7-4e6e-88be-45b368ae03e7 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:27.382911 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b74c8c-ade7-4e6e-88be-45b368ae03e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:27.406548 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-89b74c8c-ade7-4e6e-88be-45b368ae03e7 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:08:27.407778 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-89b74c8c-ade7-4e6e-88be-45b368ae03e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:27.493062 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 834/1631] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:08:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:08:27.503138 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c017055-082c-462a-b23e-e4152db526ef admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:27.504891 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-8c017055-082c-462a-b23e-e4152db526ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:27.568350 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:27.586811 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c017055-082c-462a-b23e-e4152db526ef admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:27.590624 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8bba10-5ff7-4072-bee3-408c85793b02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:27.591265 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8bba10-5ff7-4072-bee3-408c85793b02 service neutron] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:27.591548 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8bba10-5ff7-4072-bee3-408c85793b02 service neutron] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:27.591951 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8bba10-5ff7-4072-bee3-408c85793b02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:27.602125 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8bba10-5ff7-4072-bee3-408c85793b02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:27.603851 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8bba10-5ff7-4072-bee3-408c85793b02 service neutron] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:27.613786 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 798/1632] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:08:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:08:28.063490 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b74c8c-ade7-4e6e-88be-45b368ae03e7 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:28.064996 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-89b74c8c-ade7-4e6e-88be-45b368ae03e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:28.077983 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b74c8c-ade7-4e6e-88be-45b368ae03e7 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:28.084108 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6089788-f47e-4d0a-85e0-a58e725f02c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:28.084773 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6089788-f47e-4d0a-85e0-a58e725f02c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:28.085051 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6089788-f47e-4d0a-85e0-a58e725f02c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:28.085322 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6089788-f47e-4d0a-85e0-a58e725f02c6 admin admin] PATH_INFO: `/v3/projects/d2a7863d31e04296b9ee7b84ed92780b` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:28.087394 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6089788-f47e-4d0a-85e0-a58e725f02c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2a7863d31e04296b9ee7b84ed92780b)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:28.089315 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6089788-f47e-4d0a-85e0-a58e725f02c6 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:28.259359 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-e6089788-f47e-4d0a-85e0-a58e725f02c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2a7863d31e04296b9ee7b84ed92780b', 'request_id': 'req-e6089788-f47e-4d0a-85e0-a58e725f02c6'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:28.266616 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-e6089788-f47e-4d0a-85e0-a58e725f02c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2a7863d31e04296b9ee7b84ed92780b', 'request_id': 'req-e6089788-f47e-4d0a-85e0-a58e725f02c6'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:28.330339 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-e6089788-f47e-4d0a-85e0-a58e725f02c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2a7863d31e04296b9ee7b84ed92780b', 'request_id': 'req-e6089788-f47e-4d0a-85e0-a58e725f02c6'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:28.344388 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-e6089788-f47e-4d0a-85e0-a58e725f02c6 admin admin] The token cache is being invalidate because project d2a7863d31e04296b9ee7b84ed92780b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81517) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:08:28.344555 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-e6089788-f47e-4d0a-85e0-a58e725f02c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:28.345341 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-e6089788-f47e-4d0a-85e0-a58e725f02c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:28.348043 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 835/1633] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:08:28 2022] DELETE /identity/v3/projects/d2a7863d31e04296b9ee7b84ed92780b => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:08:28.351772 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8bba10-5ff7-4072-bee3-408c85793b02 service neutron] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:28.364047 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8bba10-5ff7-4072-bee3-408c85793b02 service neutron] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:28.454045 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef14e3d-cbf8-4871-8351-bd335b164189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:28.454882 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef14e3d-cbf8-4871-8351-bd335b164189 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:28.455207 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef14e3d-cbf8-4871-8351-bd335b164189 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:28.455547 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef14e3d-cbf8-4871-8351-bd335b164189 admin admin] PATH_INFO: `/v3/users/7c00170974ef4ac7b8fd6bc49769b50e` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:28.456777 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef14e3d-cbf8-4871-8351-bd335b164189 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c00170974ef4ac7b8fd6bc49769b50e)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:28.458805 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef14e3d-cbf8-4871-8351-bd335b164189 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:28.598884 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6089788-f47e-4d0a-85e0-a58e725f02c6 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:28.600744 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-e6089788-f47e-4d0a-85e0-a58e725f02c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:28.635925 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:28.673654 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6089788-f47e-4d0a-85e0-a58e725f02c6 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:28.680992 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac8d564-099b-4d6d-91fe-f5c175679b7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab599d98b0624a2e89090312bcab713f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:28.682360 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac8d564-099b-4d6d-91fe-f5c175679b7f service cinder] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:28.682646 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac8d564-099b-4d6d-91fe-f5c175679b7f service cinder] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:28.682946 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac8d564-099b-4d6d-91fe-f5c175679b7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:28.688324 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-aac8d564-099b-4d6d-91fe-f5c175679b7f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:28.693561 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-eef14e3d-cbf8-4871-8351-bd335b164189 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c00170974ef4ac7b8fd6bc49769b50e'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:28.701266 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-eef14e3d-cbf8-4871-8351-bd335b164189 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c00170974ef4ac7b8fd6bc49769b50e'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:28.714916 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-eef14e3d-cbf8-4871-8351-bd335b164189 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c00170974ef4ac7b8fd6bc49769b50e'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:28.715674 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-eef14e3d-cbf8-4871-8351-bd335b164189 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c00170974ef4ac7b8fd6bc49769b50e'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:28.756522 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac8d564-099b-4d6d-91fe-f5c175679b7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:28.758132 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac8d564-099b-4d6d-91fe-f5c175679b7f service cinder] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:28.769103 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 836/1634] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:08:28 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:08:28.783464 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-eef14e3d-cbf8-4871-8351-bd335b164189 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c00170974ef4ac7b8fd6bc49769b50e'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:28.799636 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 799/1635] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:08:28 2022] DELETE /identity/v3/users/7c00170974ef4ac7b8fd6bc49769b50e => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:08:29.024915 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac8d564-099b-4d6d-91fe-f5c175679b7f service cinder] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:29.028419 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-aac8d564-099b-4d6d-91fe-f5c175679b7f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:29.048700 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac8d564-099b-4d6d-91fe-f5c175679b7f service cinder] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:29.123451 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013b0aef-5c0f-4378-a2d7-4211627c2b74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:29.124897 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b0aef-5c0f-4378-a2d7-4211627c2b74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:29.125732 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b0aef-5c0f-4378-a2d7-4211627c2b74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:29.126182 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b0aef-5c0f-4378-a2d7-4211627c2b74 admin admin] PATH_INFO: `/v3/users/84f5fbc323c24fd88c87f8e7bb083e0a` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:29.128553 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013b0aef-5c0f-4378-a2d7-4211627c2b74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84f5fbc323c24fd88c87f8e7bb083e0a)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:29.132888 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013b0aef-5c0f-4378-a2d7-4211627c2b74 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:29.388014 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-013b0aef-5c0f-4378-a2d7-4211627c2b74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84f5fbc323c24fd88c87f8e7bb083e0a'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:29.388615 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-013b0aef-5c0f-4378-a2d7-4211627c2b74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84f5fbc323c24fd88c87f8e7bb083e0a'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:29.469133 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef14e3d-cbf8-4871-8351-bd335b164189 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:29.470606 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-eef14e3d-cbf8-4871-8351-bd335b164189 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:29.485988 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-013b0aef-5c0f-4378-a2d7-4211627c2b74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84f5fbc323c24fd88c87f8e7bb083e0a'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:29.491551 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-013b0aef-5c0f-4378-a2d7-4211627c2b74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84f5fbc323c24fd88c87f8e7bb083e0a'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:29.495109 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef14e3d-cbf8-4871-8351-bd335b164189 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:29.504854 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4b98d6-5953-4e61-b075-a1c85663eea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:29.505414 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-013b0aef-5c0f-4378-a2d7-4211627c2b74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84f5fbc323c24fd88c87f8e7bb083e0a'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:29.505950 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4b98d6-5953-4e61-b075-a1c85663eea6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:29.506393 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4b98d6-5953-4e61-b075-a1c85663eea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:29.507072 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4b98d6-5953-4e61-b075-a1c85663eea6 admin admin] PATH_INFO: `/v3/projects/f402d8a43b5d435b9ae4d6f6015bfa6f` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:29.508730 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4b98d6-5953-4e61-b075-a1c85663eea6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f402d8a43b5d435b9ae4d6f6015bfa6f)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:29.511423 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4b98d6-5953-4e61-b075-a1c85663eea6 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:29.535216 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:29.541725 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 837/1636] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:08:29 2022] DELETE /identity/v3/users/84f5fbc323c24fd88c87f8e7bb083e0a => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:08:29.546284 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b637d0f9-6a1c-442d-812a-6fd2b5eb99d5 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:29.546635 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b637d0f9-6a1c-442d-812a-6fd2b5eb99d5 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:29.546979 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b637d0f9-6a1c-442d-812a-6fd2b5eb99d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:29.578237 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-b637d0f9-6a1c-442d-812a-6fd2b5eb99d5 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:08:29.579687 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-b637d0f9-6a1c-442d-812a-6fd2b5eb99d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:29.644239 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 838/1637] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:08:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:08:29.653744 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b637d0f9-6a1c-442d-812a-6fd2b5eb99d5 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:29.656328 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-b637d0f9-6a1c-442d-812a-6fd2b5eb99d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:29.665268 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-fe4b98d6-5953-4e61-b075-a1c85663eea6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f402d8a43b5d435b9ae4d6f6015bfa6f', 'request_id': 'req-fe4b98d6-5953-4e61-b075-a1c85663eea6'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:29.677266 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-fe4b98d6-5953-4e61-b075-a1c85663eea6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f402d8a43b5d435b9ae4d6f6015bfa6f', 'request_id': 'req-fe4b98d6-5953-4e61-b075-a1c85663eea6'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:29.681998 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:29.724720 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b637d0f9-6a1c-442d-812a-6fd2b5eb99d5 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:29.771933 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-fe4b98d6-5953-4e61-b075-a1c85663eea6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f402d8a43b5d435b9ae4d6f6015bfa6f', 'request_id': 'req-fe4b98d6-5953-4e61-b075-a1c85663eea6'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:29.784648 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546cd653-c210-4833-8a63-a2118c035913 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:29.785020 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-fe4b98d6-5953-4e61-b075-a1c85663eea6 admin admin] The token cache is being invalidate because project f402d8a43b5d435b9ae4d6f6015bfa6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:08:29.785324 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-fe4b98d6-5953-4e61-b075-a1c85663eea6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:29.785591 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546cd653-c210-4833-8a63-a2118c035913 service neutron] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:29.786042 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-fe4b98d6-5953-4e61-b075-a1c85663eea6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:29.786952 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546cd653-c210-4833-8a63-a2118c035913 service neutron] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:29.786952 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546cd653-c210-4833-8a63-a2118c035913 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:29.789964 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 800/1638] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:08:29 2022] DELETE /identity/v3/projects/f402d8a43b5d435b9ae4d6f6015bfa6f => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 03:08:29.794996 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c958836d-adc4-4604-8050-8d700b7be3f7 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:29.795211 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-546cd653-c210-4833-8a63-a2118c035913 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:29.795763 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c958836d-adc4-4604-8050-8d700b7be3f7 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:29.796201 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c958836d-adc4-4604-8050-8d700b7be3f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:29.822451 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-c958836d-adc4-4604-8050-8d700b7be3f7 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:08:29.823697 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-c958836d-adc4-4604-8050-8d700b7be3f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:29.903125 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 801/1639] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:08:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:08:29.904465 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646013a9-357e-484c-ba3a-804a1418da0c None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:29.904759 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646013a9-357e-484c-ba3a-804a1418da0c None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:29.905042 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646013a9-357e-484c-ba3a-804a1418da0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:29.906748 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546cd653-c210-4833-8a63-a2118c035913 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:29.908351 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546cd653-c210-4833-8a63-a2118c035913 service neutron] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:29.920988 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 839/1640] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:08:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:08:29.928827 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-646013a9-357e-484c-ba3a-804a1418da0c None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:08:29.929597 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-646013a9-357e-484c-ba3a-804a1418da0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:29.938120 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 802/1641] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:08:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:08:29.943478 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546cd653-c210-4833-8a63-a2118c035913 service neutron] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:29.953834 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546cd653-c210-4833-8a63-a2118c035913 service neutron] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:29.959615 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fc98e6-4e37-4795-ba3f-4707d1ff49fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:29.961143 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fc98e6-4e37-4795-ba3f-4707d1ff49fd admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:29.961398 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fc98e6-4e37-4795-ba3f-4707d1ff49fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:29.961739 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fc98e6-4e37-4795-ba3f-4707d1ff49fd admin admin] PATH_INFO: `/v3/domains` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:29.962509 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fc98e6-4e37-4795-ba3f-4707d1ff49fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 03:08:29.962919 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fc98e6-4e37-4795-ba3f-4707d1ff49fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:29.964759 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fc98e6-4e37-4795-ba3f-4707d1ff49fd admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:29.981732 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 840/1642] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Oct 14 03:08:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 03:08:29.987859 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646013a9-357e-484c-ba3a-804a1418da0c None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:30.000575 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646013a9-357e-484c-ba3a-804a1418da0c None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:30.007795 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d828d5fb-7659-402d-b86c-d0aa342d1930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:30.008810 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d828d5fb-7659-402d-b86c-d0aa342d1930 admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:30.009107 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d828d5fb-7659-402d-b86c-d0aa342d1930 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:30.009391 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d828d5fb-7659-402d-b86c-d0aa342d1930 admin admin] PATH_INFO: `/v3/projects` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:30.010131 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d828d5fb-7659-402d-b86c-d0aa342d1930 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:30.012273 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d828d5fb-7659-402d-b86c-d0aa342d1930 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:30.153456 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 803/1643] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:08:29 2022] POST /identity/v3/projects => generated 381 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:08:30.157403 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fc98e6-4e37-4795-ba3f-4707d1ff49fd admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:30.183387 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fc98e6-4e37-4795-ba3f-4707d1ff49fd admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:30.259775 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967c7a89-df71-46a0-8c97-2b0e8fb13095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:30.260542 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967c7a89-df71-46a0-8c97-2b0e8fb13095 admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:30.260819 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967c7a89-df71-46a0-8c97-2b0e8fb13095 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:30.261087 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967c7a89-df71-46a0-8c97-2b0e8fb13095 admin admin] PATH_INFO: `/v3/users` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:30.261720 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967c7a89-df71-46a0-8c97-2b0e8fb13095 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:30.263755 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967c7a89-df71-46a0-8c97-2b0e8fb13095 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:30.337487 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:30.394748 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 841/1644] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:08:30 2022] POST /identity/v3/users => generated 291 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:08:30.399690 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d828d5fb-7659-402d-b86c-d0aa342d1930 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:30.412410 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d828d5fb-7659-402d-b86c-d0aa342d1930 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:30.419124 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573a51cc-c4d5-472f-9fad-fb200d9d1d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:30.420613 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573a51cc-c4d5-472f-9fad-fb200d9d1d1e admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:30.420992 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573a51cc-c4d5-472f-9fad-fb200d9d1d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:30.421314 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573a51cc-c4d5-472f-9fad-fb200d9d1d1e admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:30.422353 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573a51cc-c4d5-472f-9fad-fb200d9d1d1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:30.425237 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573a51cc-c4d5-472f-9fad-fb200d9d1d1e admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:30.440272 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 804/1645] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:08:30 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:08:30.455337 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967c7a89-df71-46a0-8c97-2b0e8fb13095 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:30.467442 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967c7a89-df71-46a0-8c97-2b0e8fb13095 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:30.473323 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6423d32d-02ba-491d-8a44-35b835577831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:30.474048 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6423d32d-02ba-491d-8a44-35b835577831 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:30.474394 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6423d32d-02ba-491d-8a44-35b835577831 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:30.474645 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6423d32d-02ba-491d-8a44-35b835577831 admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:30.475341 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6423d32d-02ba-491d-8a44-35b835577831 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:30.477236 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6423d32d-02ba-491d-8a44-35b835577831 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:30.493806 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 842/1646] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:08:30 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:08:30.506091 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573a51cc-c4d5-472f-9fad-fb200d9d1d1e admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:30.509459 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573a51cc-c4d5-472f-9fad-fb200d9d1d1e admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:30.516613 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cc2eeb-0061-44a0-a8fa-822fd3ac63a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:30.517794 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cc2eeb-0061-44a0-a8fa-822fd3ac63a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:30.518133 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cc2eeb-0061-44a0-a8fa-822fd3ac63a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:30.518414 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cc2eeb-0061-44a0-a8fa-822fd3ac63a2 admin admin] PATH_INFO: `/v3/projects/a7e0265e9e984aa4a5432f309c22f591/users/1f37721e07644fc9ad5bf95487ca06a9/roles/9bd0470b544240209a420f640c79c06c` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:30.539186 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cc2eeb-0061-44a0-a8fa-822fd3ac63a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7e0265e9e984aa4a5432f309c22f591, user_id=1f37721e07644fc9ad5bf95487ca06a9, role_id=9bd0470b544240209a420f640c79c06c)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:30.541991 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cc2eeb-0061-44a0-a8fa-822fd3ac63a2 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:30.590346 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6423d32d-02ba-491d-8a44-35b835577831 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:30.592390 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-6423d32d-02ba-491d-8a44-35b835577831 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:30.613363 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6423d32d-02ba-491d-8a44-35b835577831 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:30.617749 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 805/1647] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:08:30 2022] PUT /identity/v3/projects/a7e0265e9e984aa4a5432f309c22f591/users/1f37721e07644fc9ad5bf95487ca06a9/roles/9bd0470b544240209a420f640c79c06c => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:08:30.628728 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0448c7-2f26-4bbe-a10a-5f53995d0466 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:30.629380 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0448c7-2f26-4bbe-a10a-5f53995d0466 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:30.629887 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0448c7-2f26-4bbe-a10a-5f53995d0466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:30.677243 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-8a0448c7-2f26-4bbe-a10a-5f53995d0466 None None] MFA Rules not processed for user `1f37721e07644fc9ad5bf95487ca06a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:08:30.678656 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-8a0448c7-2f26-4bbe-a10a-5f53995d0466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:30.776838 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3831478c-59b6-4741-8a27-1e947e3395a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:30.777669 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3831478c-59b6-4741-8a27-1e947e3395a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:30.778080 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3831478c-59b6-4741-8a27-1e947e3395a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:30.778585 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3831478c-59b6-4741-8a27-1e947e3395a8 admin admin] PATH_INFO: `/v3/projects/00c0c89d0bb743f89a3f3c9ea67ef53e` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:30.779834 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3831478c-59b6-4741-8a27-1e947e3395a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00c0c89d0bb743f89a3f3c9ea67ef53e)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:30.782592 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3831478c-59b6-4741-8a27-1e947e3395a8 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:30.797368 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 806/1648] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:08:30 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 03:08:30.801543 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0448c7-2f26-4bbe-a10a-5f53995d0466 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:30.817893 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0448c7-2f26-4bbe-a10a-5f53995d0466 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:30.825487 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4621da-1a08-4868-b35b-9def2b32055e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:30.826731 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4621da-1a08-4868-b35b-9def2b32055e admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:30.827570 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4621da-1a08-4868-b35b-9def2b32055e admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:30.828081 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4621da-1a08-4868-b35b-9def2b32055e admin admin] PATH_INFO: `/v3/projects` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:30.830199 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4621da-1a08-4868-b35b-9def2b32055e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:30.832733 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4621da-1a08-4868-b35b-9def2b32055e admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:30.847641 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:30.937162 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-3831478c-59b6-4741-8a27-1e947e3395a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00c0c89d0bb743f89a3f3c9ea67ef53e', 'request_id': 'req-3831478c-59b6-4741-8a27-1e947e3395a8'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:30.943819 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-3831478c-59b6-4741-8a27-1e947e3395a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00c0c89d0bb743f89a3f3c9ea67ef53e', 'request_id': 'req-3831478c-59b6-4741-8a27-1e947e3395a8'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:30.955593 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 807/1649] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:08:30 2022] POST /identity/v3/projects => generated 379 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 03:08:30.959398 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4621da-1a08-4868-b35b-9def2b32055e admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:31.031093 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4621da-1a08-4868-b35b-9def2b32055e admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:31.036079 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48d79eb-c607-4a2e-9ec1-c6d1021dc064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:31.036822 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d79eb-c607-4a2e-9ec1-c6d1021dc064 admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:31.037267 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d79eb-c607-4a2e-9ec1-c6d1021dc064 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:31.037854 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d79eb-c607-4a2e-9ec1-c6d1021dc064 admin admin] PATH_INFO: `/v3/users` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:31.039222 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48d79eb-c607-4a2e-9ec1-c6d1021dc064 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:31.041188 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48d79eb-c607-4a2e-9ec1-c6d1021dc064 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:31.050699 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-3831478c-59b6-4741-8a27-1e947e3395a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00c0c89d0bb743f89a3f3c9ea67ef53e', 'request_id': 'req-3831478c-59b6-4741-8a27-1e947e3395a8'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:31.060941 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:31.065493 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-3831478c-59b6-4741-8a27-1e947e3395a8 admin admin] The token cache is being invalidate because project 00c0c89d0bb743f89a3f3c9ea67ef53e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81517) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:08:31.065887 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-3831478c-59b6-4741-8a27-1e947e3395a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:31.068769 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-3831478c-59b6-4741-8a27-1e947e3395a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:31.071604 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 843/1650] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:08:30 2022] DELETE /identity/v3/projects/00c0c89d0bb743f89a3f3c9ea67ef53e => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 03:08:31.073132 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3831478c-59b6-4741-8a27-1e947e3395a8 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:31.075044 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-3831478c-59b6-4741-8a27-1e947e3395a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:31.115609 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 808/1651] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:08:30 2022] POST /identity/v3/users => generated 296 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 03:08:31.120926 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48d79eb-c607-4a2e-9ec1-c6d1021dc064 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:31.122518 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-f48d79eb-c607-4a2e-9ec1-c6d1021dc064 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:31.144801 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48d79eb-c607-4a2e-9ec1-c6d1021dc064 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:31.150559 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dd67b1-030f-477f-ab15-942ef66f1672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:31.151326 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dd67b1-030f-477f-ab15-942ef66f1672 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:31.152051 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dd67b1-030f-477f-ab15-942ef66f1672 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:31.152051 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dd67b1-030f-477f-ab15-942ef66f1672 admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:31.152891 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dd67b1-030f-477f-ab15-942ef66f1672 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:31.156282 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dd67b1-030f-477f-ab15-942ef66f1672 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:31.182648 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3831478c-59b6-4741-8a27-1e947e3395a8 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:31.189387 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 809/1652] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:08:31 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:08:31.193193 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dd67b1-030f-477f-ab15-942ef66f1672 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:31.201148 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bf21db-974b-415b-8970-463384c87163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:31.201148 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bf21db-974b-415b-8970-463384c87163 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:31.201571 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bf21db-974b-415b-8970-463384c87163 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:31.202018 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bf21db-974b-415b-8970-463384c87163 admin admin] PATH_INFO: `/v3/users/9a22abbb2e344596a0c33c26e82fb741` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:31.203396 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bf21db-974b-415b-8970-463384c87163 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a22abbb2e344596a0c33c26e82fb741)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:31.205630 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bf21db-974b-415b-8970-463384c87163 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:31.209527 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dd67b1-030f-477f-ab15-942ef66f1672 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:31.217465 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b583a10f-bd52-41ab-91a3-74f0dd37d6da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:31.219024 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b583a10f-bd52-41ab-91a3-74f0dd37d6da admin admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:31.219552 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b583a10f-bd52-41ab-91a3-74f0dd37d6da admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:31.219841 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b583a10f-bd52-41ab-91a3-74f0dd37d6da admin admin] PATH_INFO: `/v3/projects/83f9b422ab7b466e8b88aa0a4f2c04c3/users/ee3cd32b940b4d73bf000e04a6fc44a3/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:31.244357 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b583a10f-bd52-41ab-91a3-74f0dd37d6da admin admin] RBAC: Authorizing `identity:create_grant(project_id=83f9b422ab7b466e8b88aa0a4f2c04c3, user_id=ee3cd32b940b4d73bf000e04a6fc44a3, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:31.246828 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b583a10f-bd52-41ab-91a3-74f0dd37d6da admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:31.370816 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 810/1653] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:08:31 2022] PUT /identity/v3/projects/83f9b422ab7b466e8b88aa0a4f2c04c3/users/ee3cd32b940b4d73bf000e04a6fc44a3/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:08:31.375468 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b583a10f-bd52-41ab-91a3-74f0dd37d6da admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:31.435633 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-27bf21db-974b-415b-8970-463384c87163 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a22abbb2e344596a0c33c26e82fb741'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:31.436629 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-27bf21db-974b-415b-8970-463384c87163 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a22abbb2e344596a0c33c26e82fb741'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:31.458264 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b583a10f-bd52-41ab-91a3-74f0dd37d6da admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:31.464294 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ffc345-8a52-43df-bf48-dc8b4c2bb943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:31.464810 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ffc345-8a52-43df-bf48-dc8b4c2bb943 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:31.465098 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ffc345-8a52-43df-bf48-dc8b4c2bb943 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:31.465374 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ffc345-8a52-43df-bf48-dc8b4c2bb943 admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:31.466302 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ffc345-8a52-43df-bf48-dc8b4c2bb943 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:31.468355 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ffc345-8a52-43df-bf48-dc8b4c2bb943 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:31.482368 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 811/1654] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:08:31 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 03:08:31.486358 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ffc345-8a52-43df-bf48-dc8b4c2bb943 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:31.488498 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-79ffc345-8a52-43df-bf48-dc8b4c2bb943 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:31.508348 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-27bf21db-974b-415b-8970-463384c87163 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a22abbb2e344596a0c33c26e82fb741'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:31.508998 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-27bf21db-974b-415b-8970-463384c87163 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a22abbb2e344596a0c33c26e82fb741'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:31.517420 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ffc345-8a52-43df-bf48-dc8b4c2bb943 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:31.520076 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-27bf21db-974b-415b-8970-463384c87163 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a22abbb2e344596a0c33c26e82fb741'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:31.528409 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6c6ac4-8f38-4a82-95a9-08b61734d0fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:31.529837 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6c6ac4-8f38-4a82-95a9-08b61734d0fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:31.530307 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6c6ac4-8f38-4a82-95a9-08b61734d0fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:31.534327 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6c6ac4-8f38-4a82-95a9-08b61734d0fc admin admin] PATH_INFO: `/v3/projects/83f9b422ab7b466e8b88aa0a4f2c04c3/users/ee3cd32b940b4d73bf000e04a6fc44a3/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:31.539780 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6c6ac4-8f38-4a82-95a9-08b61734d0fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=83f9b422ab7b466e8b88aa0a4f2c04c3, user_id=ee3cd32b940b4d73bf000e04a6fc44a3, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:31.542877 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6c6ac4-8f38-4a82-95a9-08b61734d0fc admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:31.550899 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 844/1655] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:08:31 2022] DELETE /identity/v3/users/9a22abbb2e344596a0c33c26e82fb741 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:08:31.563395 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 812/1656] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:08:31 2022] PUT /identity/v3/projects/83f9b422ab7b466e8b88aa0a4f2c04c3/users/ee3cd32b940b4d73bf000e04a6fc44a3/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 03:08:31.573341 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2ee045-f8f2-4542-b4a1-fd2b7337b538 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:31.573739 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2ee045-f8f2-4542-b4a1-fd2b7337b538 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:31.574090 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2ee045-f8f2-4542-b4a1-fd2b7337b538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:31.611496 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-9f2ee045-f8f2-4542-b4a1-fd2b7337b538 None None] MFA Rules not processed for user `ee3cd32b940b4d73bf000e04a6fc44a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:08:31.612938 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-9f2ee045-f8f2-4542-b4a1-fd2b7337b538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:31.700147 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 845/1657] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:08:31 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:08:31.712642 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6c6ac4-8f38-4a82-95a9-08b61734d0fc admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:31.715379 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-dd6c6ac4-8f38-4a82-95a9-08b61734d0fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:31.750939 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:31.783697 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6c6ac4-8f38-4a82-95a9-08b61734d0fc admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:31.787827 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8ed188-f9e6-4e8f-ab5f-50cba82f96e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab599d98b0624a2e89090312bcab713f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:31.788486 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8ed188-f9e6-4e8f-ab5f-50cba82f96e0 service cinder] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:31.788796 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8ed188-f9e6-4e8f-ab5f-50cba82f96e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:31.789073 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8ed188-f9e6-4e8f-ab5f-50cba82f96e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:31.799082 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8ed188-f9e6-4e8f-ab5f-50cba82f96e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:31.800523 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8ed188-f9e6-4e8f-ab5f-50cba82f96e0 service cinder] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:31.811533 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 813/1658] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:08:31 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:08:32.227684 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2ee045-f8f2-4542-b4a1-fd2b7337b538 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:32.231819 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-9f2ee045-f8f2-4542-b4a1-fd2b7337b538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:32.247650 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:32.452272 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2ee045-f8f2-4542-b4a1-fd2b7337b538 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:32.463250 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073c57c7-b880-4c97-8ace-d909d26e3f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:32.463888 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073c57c7-b880-4c97-8ace-d909d26e3f9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:32.464224 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073c57c7-b880-4c97-8ace-d909d26e3f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:32.464889 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073c57c7-b880-4c97-8ace-d909d26e3f9f admin admin] PATH_INFO: `/v3/projects/89fadb3a6cb54d599fddef24a3eb945e` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:32.468092 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073c57c7-b880-4c97-8ace-d909d26e3f9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=89fadb3a6cb54d599fddef24a3eb945e)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:32.470097 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073c57c7-b880-4c97-8ace-d909d26e3f9f admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:32.621243 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-073c57c7-b880-4c97-8ace-d909d26e3f9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89fadb3a6cb54d599fddef24a3eb945e', 'request_id': 'req-073c57c7-b880-4c97-8ace-d909d26e3f9f'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:32.625655 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-073c57c7-b880-4c97-8ace-d909d26e3f9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89fadb3a6cb54d599fddef24a3eb945e', 'request_id': 'req-073c57c7-b880-4c97-8ace-d909d26e3f9f'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:32.755659 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-073c57c7-b880-4c97-8ace-d909d26e3f9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89fadb3a6cb54d599fddef24a3eb945e', 'request_id': 'req-073c57c7-b880-4c97-8ace-d909d26e3f9f'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:32.776615 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-073c57c7-b880-4c97-8ace-d909d26e3f9f admin admin] The token cache is being invalidate because project 89fadb3a6cb54d599fddef24a3eb945e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81517) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:08:32.776971 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-073c57c7-b880-4c97-8ace-d909d26e3f9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:32.778316 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-073c57c7-b880-4c97-8ace-d909d26e3f9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:32.784127 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 846/1659] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:08:32 2022] DELETE /identity/v3/projects/89fadb3a6cb54d599fddef24a3eb945e => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:08:32.785245 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8ed188-f9e6-4e8f-ab5f-50cba82f96e0 service cinder] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:32.796763 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8ed188-f9e6-4e8f-ab5f-50cba82f96e0 service cinder] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:32.874153 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96ebc77-016f-4c13-bd98-07d6c65c5f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:32.874949 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96ebc77-016f-4c13-bd98-07d6c65c5f7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:32.875235 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96ebc77-016f-4c13-bd98-07d6c65c5f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:32.875549 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96ebc77-016f-4c13-bd98-07d6c65c5f7a admin admin] PATH_INFO: `/v3/users/8397903bcaa140899202b00cd0e39ec2` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:32.877994 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96ebc77-016f-4c13-bd98-07d6c65c5f7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8397903bcaa140899202b00cd0e39ec2)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:32.880940 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96ebc77-016f-4c13-bd98-07d6c65c5f7a admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:33.112278 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-c96ebc77-016f-4c13-bd98-07d6c65c5f7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8397903bcaa140899202b00cd0e39ec2'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:33.119641 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-c96ebc77-016f-4c13-bd98-07d6c65c5f7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8397903bcaa140899202b00cd0e39ec2'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:33.133419 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-c96ebc77-016f-4c13-bd98-07d6c65c5f7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8397903bcaa140899202b00cd0e39ec2'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:33.134274 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-c96ebc77-016f-4c13-bd98-07d6c65c5f7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8397903bcaa140899202b00cd0e39ec2'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:33.215580 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-c96ebc77-016f-4c13-bd98-07d6c65c5f7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8397903bcaa140899202b00cd0e39ec2'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:33.222188 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 814/1660] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:08:32 2022] DELETE /identity/v3/users/8397903bcaa140899202b00cd0e39ec2 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:08:33.636205 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073c57c7-b880-4c97-8ace-d909d26e3f9f admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:33.638145 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-073c57c7-b880-4c97-8ace-d909d26e3f9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:33.642966 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96ebc77-016f-4c13-bd98-07d6c65c5f7a admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:33.650795 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-c96ebc77-016f-4c13-bd98-07d6c65c5f7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:33.669658 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073c57c7-b880-4c97-8ace-d909d26e3f9f admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:33.768686 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:33.789448 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f9175f-2579-4f9f-8133-c7a7c654885e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:33.790489 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f9175f-2579-4f9f-8133-c7a7c654885e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:33.790925 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f9175f-2579-4f9f-8133-c7a7c654885e admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:33.791359 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f9175f-2579-4f9f-8133-c7a7c654885e admin admin] PATH_INFO: `/v3/projects/7845ac7ad3bc40b693af52ba81517309` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:33.794883 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f9175f-2579-4f9f-8133-c7a7c654885e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7845ac7ad3bc40b693af52ba81517309)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:33.797281 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96ebc77-016f-4c13-bd98-07d6c65c5f7a admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:33.802564 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f9175f-2579-4f9f-8133-c7a7c654885e admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:33.809788 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a04d507-660b-4f31-a7ed-0946592e87ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:33.810450 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a04d507-660b-4f31-a7ed-0946592e87ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:33.811895 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a04d507-660b-4f31-a7ed-0946592e87ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:33.811895 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a04d507-660b-4f31-a7ed-0946592e87ef admin admin] PATH_INFO: `/v3/users/1f37721e07644fc9ad5bf95487ca06a9` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:33.812307 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a04d507-660b-4f31-a7ed-0946592e87ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f37721e07644fc9ad5bf95487ca06a9)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:33.815664 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a04d507-660b-4f31-a7ed-0946592e87ef admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:33.993812 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:34.007968 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-2a04d507-660b-4f31-a7ed-0946592e87ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f37721e07644fc9ad5bf95487ca06a9'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:34.017424 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-b2f9175f-2579-4f9f-8133-c7a7c654885e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7845ac7ad3bc40b693af52ba81517309', 'request_id': 'req-b2f9175f-2579-4f9f-8133-c7a7c654885e'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:34.020180 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-2a04d507-660b-4f31-a7ed-0946592e87ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f37721e07644fc9ad5bf95487ca06a9'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:34.026010 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-b2f9175f-2579-4f9f-8133-c7a7c654885e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7845ac7ad3bc40b693af52ba81517309', 'request_id': 'req-b2f9175f-2579-4f9f-8133-c7a7c654885e'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:34.039506 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-2a04d507-660b-4f31-a7ed-0946592e87ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f37721e07644fc9ad5bf95487ca06a9'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:34.040287 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-2a04d507-660b-4f31-a7ed-0946592e87ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f37721e07644fc9ad5bf95487ca06a9'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:34.125404 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-b2f9175f-2579-4f9f-8133-c7a7c654885e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7845ac7ad3bc40b693af52ba81517309', 'request_id': 'req-b2f9175f-2579-4f9f-8133-c7a7c654885e'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:34.138771 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-b2f9175f-2579-4f9f-8133-c7a7c654885e admin admin] The token cache is being invalidate because project 7845ac7ad3bc40b693af52ba81517309 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81517) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:08:34.139189 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-b2f9175f-2579-4f9f-8133-c7a7c654885e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:34.140931 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-b2f9175f-2579-4f9f-8133-c7a7c654885e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:34.147181 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 847/1661] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:08:33 2022] DELETE /identity/v3/projects/7845ac7ad3bc40b693af52ba81517309 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:08:34.147181 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f9175f-2579-4f9f-8133-c7a7c654885e admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:34.147181 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-b2f9175f-2579-4f9f-8133-c7a7c654885e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:34.181900 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-2a04d507-660b-4f31-a7ed-0946592e87ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f37721e07644fc9ad5bf95487ca06a9'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:34.190543 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 815/1662] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:08:33 2022] DELETE /identity/v3/users/1f37721e07644fc9ad5bf95487ca06a9 => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:08:34.193050 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9eb031-031f-4f7b-81c2-0e1ab1db423c None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:34.193411 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9eb031-031f-4f7b-81c2-0e1ab1db423c None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:34.193698 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9eb031-031f-4f7b-81c2-0e1ab1db423c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:34.218053 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-9a9eb031-031f-4f7b-81c2-0e1ab1db423c None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:08:34.219369 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-9a9eb031-031f-4f7b-81c2-0e1ab1db423c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:34.228864 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f9175f-2579-4f9f-8133-c7a7c654885e admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:34.289450 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 816/1663] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:08:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:08:34.291526 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9b453c-4756-41f2-80d8-f62cbdb908ab None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:34.292056 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9b453c-4756-41f2-80d8-f62cbdb908ab None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:34.292545 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9b453c-4756-41f2-80d8-f62cbdb908ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:34.319988 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-5b9b453c-4756-41f2-80d8-f62cbdb908ab None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:08:34.321236 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-5b9b453c-4756-41f2-80d8-f62cbdb908ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:34.321377 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bc555f-7a3b-4246-88c9-073716de667b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:34.322077 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc555f-7a3b-4246-88c9-073716de667b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:34.322362 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc555f-7a3b-4246-88c9-073716de667b admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:34.322641 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc555f-7a3b-4246-88c9-073716de667b admin admin] PATH_INFO: `/v3/users/eca194f38abd4097b5e614983476820c` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:34.323781 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bc555f-7a3b-4246-88c9-073716de667b admin admin] RBAC: Authorizing `identity:delete_user(user_id=eca194f38abd4097b5e614983476820c)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:34.325812 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bc555f-7a3b-4246-88c9-073716de667b admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:34.331309 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 817/1664] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:08:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:08:34.332798 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254eb222-190b-40e5-9f81-651ab711832a None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:34.333153 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254eb222-190b-40e5-9f81-651ab711832a None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:34.333495 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254eb222-190b-40e5-9f81-651ab711832a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:34.354802 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-254eb222-190b-40e5-9f81-651ab711832a None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:08:34.356567 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-254eb222-190b-40e5-9f81-651ab711832a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:34.364939 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 818/1665] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:08:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:08:34.365876 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254eb222-190b-40e5-9f81-651ab711832a None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:34.367364 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-254eb222-190b-40e5-9f81-651ab711832a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:34.449142 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254eb222-190b-40e5-9f81-651ab711832a None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:34.453276 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62cc8ff-e69a-440a-93c7-356d91586445 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:34.454239 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62cc8ff-e69a-440a-93c7-356d91586445 service neutron] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:34.454918 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62cc8ff-e69a-440a-93c7-356d91586445 service neutron] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:34.454918 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62cc8ff-e69a-440a-93c7-356d91586445 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:34.467339 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62cc8ff-e69a-440a-93c7-356d91586445 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:34.469187 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62cc8ff-e69a-440a-93c7-356d91586445 service neutron] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:34.474048 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:34.484884 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 819/1666] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:08:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:08:34.485991 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62cc8ff-e69a-440a-93c7-356d91586445 service neutron] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:34.498695 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62cc8ff-e69a-440a-93c7-356d91586445 service neutron] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:34.504625 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd783fb7-617d-4b1f-82b0-214e7265b0db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:34.505307 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd783fb7-617d-4b1f-82b0-214e7265b0db admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:34.505643 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd783fb7-617d-4b1f-82b0-214e7265b0db admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:34.505957 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd783fb7-617d-4b1f-82b0-214e7265b0db admin admin] PATH_INFO: `/v3/domains` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:34.507694 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd783fb7-617d-4b1f-82b0-214e7265b0db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 03:08:34.508007 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd783fb7-617d-4b1f-82b0-214e7265b0db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:34.510241 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd783fb7-617d-4b1f-82b0-214e7265b0db admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:34.515207 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-13bc555f-7a3b-4246-88c9-073716de667b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eca194f38abd4097b5e614983476820c'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:34.518725 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-13bc555f-7a3b-4246-88c9-073716de667b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eca194f38abd4097b5e614983476820c'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:34.536132 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 820/1667] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Oct 14 03:08:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 03:08:34.537133 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd783fb7-617d-4b1f-82b0-214e7265b0db admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:34.541919 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-cd783fb7-617d-4b1f-82b0-214e7265b0db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:34.570200 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd783fb7-617d-4b1f-82b0-214e7265b0db admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:34.579091 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603963cf-8688-4869-a006-88f7d4ed11ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:34.579896 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603963cf-8688-4869-a006-88f7d4ed11ec admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:34.580186 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603963cf-8688-4869-a006-88f7d4ed11ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:34.580686 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603963cf-8688-4869-a006-88f7d4ed11ec admin admin] PATH_INFO: `/v3/projects` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:34.585949 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603963cf-8688-4869-a006-88f7d4ed11ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:34.585949 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603963cf-8688-4869-a006-88f7d4ed11ec admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:34.604516 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-13bc555f-7a3b-4246-88c9-073716de667b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eca194f38abd4097b5e614983476820c'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:34.605171 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-13bc555f-7a3b-4246-88c9-073716de667b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eca194f38abd4097b5e614983476820c'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:34.614658 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-13bc555f-7a3b-4246-88c9-073716de667b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eca194f38abd4097b5e614983476820c'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:34.626394 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:34.642027 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 848/1668] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:08:34 2022] DELETE /identity/v3/users/eca194f38abd4097b5e614983476820c => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:08:34.644230 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf2a533-bdd7-456a-876e-f900eaf097ff None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:34.646077 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf2a533-bdd7-456a-876e-f900eaf097ff None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:34.646588 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf2a533-bdd7-456a-876e-f900eaf097ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:34.674187 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-0bf2a533-bdd7-456a-876e-f900eaf097ff None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:08:34.675481 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-0bf2a533-bdd7-456a-876e-f900eaf097ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:34.703838 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 821/1669] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:08:34 2022] POST /identity/v3/projects => generated 381 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:08:34.709088 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603963cf-8688-4869-a006-88f7d4ed11ec admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:34.762565 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 849/1670] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:08:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:08:34.779952 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf2a533-bdd7-456a-876e-f900eaf097ff None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:34.782492 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-0bf2a533-bdd7-456a-876e-f900eaf097ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:34.839592 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603963cf-8688-4869-a006-88f7d4ed11ec admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:34.847312 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265f4f14-7bb0-44ae-a036-d1a846914f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:34.849013 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265f4f14-7bb0-44ae-a036-d1a846914f9c admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:34.849424 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265f4f14-7bb0-44ae-a036-d1a846914f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:34.850070 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265f4f14-7bb0-44ae-a036-d1a846914f9c admin admin] PATH_INFO: `/v3/users` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:34.851377 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265f4f14-7bb0-44ae-a036-d1a846914f9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:34.854346 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265f4f14-7bb0-44ae-a036-d1a846914f9c admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:34.863347 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf2a533-bdd7-456a-876e-f900eaf097ff None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:34.928418 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2e6a65-5efa-4a89-992d-c0c27205eba4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:34.929219 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2e6a65-5efa-4a89-992d-c0c27205eba4 service neutron] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:34.929706 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2e6a65-5efa-4a89-992d-c0c27205eba4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:34.931367 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2e6a65-5efa-4a89-992d-c0c27205eba4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:34.945856 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2e6a65-5efa-4a89-992d-c0c27205eba4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:34.947358 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2e6a65-5efa-4a89-992d-c0c27205eba4 service neutron] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:34.963754 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 822/1671] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:08:34 2022] POST /identity/v3/users => generated 291 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:08:34.967985 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265f4f14-7bb0-44ae-a036-d1a846914f9c admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:34.981569 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 850/1672] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:08:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 03:08:34.987018 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265f4f14-7bb0-44ae-a036-d1a846914f9c admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:34.993568 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c605d7-5c2a-4668-aa03-555fe2838783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:34.994298 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c605d7-5c2a-4668-aa03-555fe2838783 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:34.994838 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c605d7-5c2a-4668-aa03-555fe2838783 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:34.995193 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c605d7-5c2a-4668-aa03-555fe2838783 admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:34.995986 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c605d7-5c2a-4668-aa03-555fe2838783 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:34.998059 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c605d7-5c2a-4668-aa03-555fe2838783 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:35.014676 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 823/1673] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:08:34 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 03:08:35.018422 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2e6a65-5efa-4a89-992d-c0c27205eba4 service neutron] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:35.031081 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2e6a65-5efa-4a89-992d-c0c27205eba4 service neutron] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:35.040291 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253be35c-ef26-4d45-bfae-1f3b47c9a2bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:35.046082 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253be35c-ef26-4d45-bfae-1f3b47c9a2bb admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:35.046082 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253be35c-ef26-4d45-bfae-1f3b47c9a2bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:35.046082 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253be35c-ef26-4d45-bfae-1f3b47c9a2bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:35.046082 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253be35c-ef26-4d45-bfae-1f3b47c9a2bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:35.049214 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253be35c-ef26-4d45-bfae-1f3b47c9a2bb admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:35.079764 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 851/1674] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:08:35 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 03:08:35.085027 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c605d7-5c2a-4668-aa03-555fe2838783 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:35.098287 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c605d7-5c2a-4668-aa03-555fe2838783 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:35.111094 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0550c1-b070-467c-987e-2302273b79b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:35.111813 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0550c1-b070-467c-987e-2302273b79b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:35.112094 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0550c1-b070-467c-987e-2302273b79b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:35.112367 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0550c1-b070-467c-987e-2302273b79b1 admin admin] PATH_INFO: `/v3/projects/ff08703801ed4e03a584956222704d4a/users/ce924de479c44547a27ea09a3de9e980/roles/9bd0470b544240209a420f640c79c06c` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:35.152637 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0550c1-b070-467c-987e-2302273b79b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff08703801ed4e03a584956222704d4a, user_id=ce924de479c44547a27ea09a3de9e980, role_id=9bd0470b544240209a420f640c79c06c)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:35.155447 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0550c1-b070-467c-987e-2302273b79b1 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:35.289642 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 824/1675] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:08:35 2022] PUT /identity/v3/projects/ff08703801ed4e03a584956222704d4a/users/ce924de479c44547a27ea09a3de9e980/roles/9bd0470b544240209a420f640c79c06c => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:08:35.299965 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e73515-a2f3-4a8e-9806-d0c7eda759b2 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:35.300146 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e73515-a2f3-4a8e-9806-d0c7eda759b2 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:35.300435 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e73515-a2f3-4a8e-9806-d0c7eda759b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:35.369876 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-56e73515-a2f3-4a8e-9806-d0c7eda759b2 None None] MFA Rules not processed for user `ce924de479c44547a27ea09a3de9e980`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:08:35.369876 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-56e73515-a2f3-4a8e-9806-d0c7eda759b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:35.411562 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:35.467212 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 852/1676] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:08:35 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:08:35.472105 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0550c1-b070-467c-987e-2302273b79b1 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:35.562380 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0550c1-b070-467c-987e-2302273b79b1 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:35.568823 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a58e7f5-1913-4268-927e-253ec9b806b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:35.569080 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a58e7f5-1913-4268-927e-253ec9b806b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:35.569262 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a58e7f5-1913-4268-927e-253ec9b806b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:35.569577 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a58e7f5-1913-4268-927e-253ec9b806b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:35.570721 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a58e7f5-1913-4268-927e-253ec9b806b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:35.572536 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a58e7f5-1913-4268-927e-253ec9b806b8 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:35.692105 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 825/1677] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:08:35 2022] POST /identity/v3/projects => generated 381 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:08:35.696032 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e73515-a2f3-4a8e-9806-d0c7eda759b2 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:35.713286 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e73515-a2f3-4a8e-9806-d0c7eda759b2 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:35.819237 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae28173-7a44-45e6-9b6a-b884279add4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:35.820069 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae28173-7a44-45e6-9b6a-b884279add4d admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:35.820472 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae28173-7a44-45e6-9b6a-b884279add4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:35.820841 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae28173-7a44-45e6-9b6a-b884279add4d admin admin] PATH_INFO: `/v3/users` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:35.821577 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae28173-7a44-45e6-9b6a-b884279add4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:35.824081 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae28173-7a44-45e6-9b6a-b884279add4d admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:35.925300 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 853/1678] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:08:35 2022] POST /identity/v3/users => generated 297 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:08:35.929366 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a58e7f5-1913-4268-927e-253ec9b806b8 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:35.935937 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:35.941427 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a58e7f5-1913-4268-927e-253ec9b806b8 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:35.951180 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44b35f6-5dbf-4aa0-8ab7-8a9dfe4ab17f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:35.951180 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44b35f6-5dbf-4aa0-8ab7-8a9dfe4ab17f admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:35.951180 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44b35f6-5dbf-4aa0-8ab7-8a9dfe4ab17f admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:35.951180 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44b35f6-5dbf-4aa0-8ab7-8a9dfe4ab17f admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:35.951639 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44b35f6-5dbf-4aa0-8ab7-8a9dfe4ab17f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:35.951639 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44b35f6-5dbf-4aa0-8ab7-8a9dfe4ab17f admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:35.961108 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 826/1679] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:08:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 03:08:35.965223 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae28173-7a44-45e6-9b6a-b884279add4d admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:35.975306 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae28173-7a44-45e6-9b6a-b884279add4d admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:35.980659 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392413f8-8658-446a-9450-689dad628b72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:35.981316 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392413f8-8658-446a-9450-689dad628b72 admin admin] REQUEST_METHOD: `PUT` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:35.981651 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392413f8-8658-446a-9450-689dad628b72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:35.981963 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392413f8-8658-446a-9450-689dad628b72 admin admin] PATH_INFO: `/v3/projects/fb4d940eede741469b90aa1b4d0cd4b3/users/e7d758dd576a4c35bee4e3b1d6c24029/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:35.997570 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392413f8-8658-446a-9450-689dad628b72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb4d940eede741469b90aa1b4d0cd4b3, user_id=e7d758dd576a4c35bee4e3b1d6c24029, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:36.003170 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392413f8-8658-446a-9450-689dad628b72 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:36.006694 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44b35f6-5dbf-4aa0-8ab7-8a9dfe4ab17f admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:36.008513 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-e44b35f6-5dbf-4aa0-8ab7-8a9dfe4ab17f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:36.026994 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44b35f6-5dbf-4aa0-8ab7-8a9dfe4ab17f admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:36.033604 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015a2101-3ff6-4dfe-acb2-411f657a6d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:36.034504 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015a2101-3ff6-4dfe-acb2-411f657a6d69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:36.035512 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015a2101-3ff6-4dfe-acb2-411f657a6d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:36.036228 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015a2101-3ff6-4dfe-acb2-411f657a6d69 admin admin] PATH_INFO: `/v3/projects/a7e0265e9e984aa4a5432f309c22f591` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:36.037783 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015a2101-3ff6-4dfe-acb2-411f657a6d69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7e0265e9e984aa4a5432f309c22f591)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:36.040247 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015a2101-3ff6-4dfe-acb2-411f657a6d69 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:36.096479 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 854/1680] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:08:35 2022] PUT /identity/v3/projects/fb4d940eede741469b90aa1b4d0cd4b3/users/e7d758dd576a4c35bee4e3b1d6c24029/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:08:36.100470 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392413f8-8658-446a-9450-689dad628b72 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:36.102190 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-392413f8-8658-446a-9450-689dad628b72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:36.160439 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-015a2101-3ff6-4dfe-acb2-411f657a6d69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7e0265e9e984aa4a5432f309c22f591', 'request_id': 'req-015a2101-3ff6-4dfe-acb2-411f657a6d69'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:36.174558 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392413f8-8658-446a-9450-689dad628b72 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:36.178468 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-015a2101-3ff6-4dfe-acb2-411f657a6d69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7e0265e9e984aa4a5432f309c22f591', 'request_id': 'req-015a2101-3ff6-4dfe-acb2-411f657a6d69'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:36.219796 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:36.222167 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-015a2101-3ff6-4dfe-acb2-411f657a6d69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7e0265e9e984aa4a5432f309c22f591', 'request_id': 'req-015a2101-3ff6-4dfe-acb2-411f657a6d69'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:36.237654 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-015a2101-3ff6-4dfe-acb2-411f657a6d69 admin admin] The token cache is being invalidate because project a7e0265e9e984aa4a5432f309c22f591 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:08:36.238341 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-015a2101-3ff6-4dfe-acb2-411f657a6d69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:36.240622 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-015a2101-3ff6-4dfe-acb2-411f657a6d69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:36.242375 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 827/1681] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:08:36 2022] DELETE /identity/v3/projects/a7e0265e9e984aa4a5432f309c22f591 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 03:08:36.245181 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015a2101-3ff6-4dfe-acb2-411f657a6d69 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:36.245181 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-015a2101-3ff6-4dfe-acb2-411f657a6d69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:36.252113 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5d2be2-a6fc-43c9-a9ba-c98e8179358a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:36.252757 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5d2be2-a6fc-43c9-a9ba-c98e8179358a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:36.258043 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5d2be2-a6fc-43c9-a9ba-c98e8179358a admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:36.259612 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5d2be2-a6fc-43c9-a9ba-c98e8179358a admin admin] PATH_INFO: `/v3/projects/df8d43e298c5449f8f0e8c3db7c07a41` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:36.262458 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5d2be2-a6fc-43c9-a9ba-c98e8179358a admin admin] RBAC: Authorizing `identity:delete_project(project_id=df8d43e298c5449f8f0e8c3db7c07a41)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:36.263634 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015a2101-3ff6-4dfe-acb2-411f657a6d69 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:36.265947 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5d2be2-a6fc-43c9-a9ba-c98e8179358a admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:36.270298 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9581ee-aec6-46f5-89b8-67ebaf38ae97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:36.270873 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9581ee-aec6-46f5-89b8-67ebaf38ae97 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:36.275897 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9581ee-aec6-46f5-89b8-67ebaf38ae97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:36.277215 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9581ee-aec6-46f5-89b8-67ebaf38ae97 admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:36.277992 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9581ee-aec6-46f5-89b8-67ebaf38ae97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:36.282912 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9581ee-aec6-46f5-89b8-67ebaf38ae97 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:36.303466 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 828/1682] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:08:36 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:08:36.303466 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9581ee-aec6-46f5-89b8-67ebaf38ae97 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:36.304427 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-4c9581ee-aec6-46f5-89b8-67ebaf38ae97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:36.322601 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9581ee-aec6-46f5-89b8-67ebaf38ae97 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:36.327678 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bb08e0-4e7e-408b-bfd9-d22cc3bc1be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:36.328647 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bb08e0-4e7e-408b-bfd9-d22cc3bc1be3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:36.329034 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bb08e0-4e7e-408b-bfd9-d22cc3bc1be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:36.329491 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bb08e0-4e7e-408b-bfd9-d22cc3bc1be3 admin admin] PATH_INFO: `/v3/users/ee3cd32b940b4d73bf000e04a6fc44a3` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:36.330817 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bb08e0-4e7e-408b-bfd9-d22cc3bc1be3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee3cd32b940b4d73bf000e04a6fc44a3)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:36.333000 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bb08e0-4e7e-408b-bfd9-d22cc3bc1be3 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:36.405589 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-8a5d2be2-a6fc-43c9-a9ba-c98e8179358a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df8d43e298c5449f8f0e8c3db7c07a41', 'request_id': 'req-8a5d2be2-a6fc-43c9-a9ba-c98e8179358a'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:36.412848 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-8a5d2be2-a6fc-43c9-a9ba-c98e8179358a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df8d43e298c5449f8f0e8c3db7c07a41', 'request_id': 'req-8a5d2be2-a6fc-43c9-a9ba-c98e8179358a'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:36.495609 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-e6bb08e0-4e7e-408b-bfd9-d22cc3bc1be3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee3cd32b940b4d73bf000e04a6fc44a3'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:36.503445 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-e6bb08e0-4e7e-408b-bfd9-d22cc3bc1be3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee3cd32b940b4d73bf000e04a6fc44a3'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:36.503609 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-8a5d2be2-a6fc-43c9-a9ba-c98e8179358a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df8d43e298c5449f8f0e8c3db7c07a41', 'request_id': 'req-8a5d2be2-a6fc-43c9-a9ba-c98e8179358a'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:36.517648 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-e6bb08e0-4e7e-408b-bfd9-d22cc3bc1be3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee3cd32b940b4d73bf000e04a6fc44a3'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:36.518750 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-e6bb08e0-4e7e-408b-bfd9-d22cc3bc1be3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee3cd32b940b4d73bf000e04a6fc44a3'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:36.519191 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-8a5d2be2-a6fc-43c9-a9ba-c98e8179358a admin admin] The token cache is being invalidate because project df8d43e298c5449f8f0e8c3db7c07a41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81517) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:08:36.519492 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-8a5d2be2-a6fc-43c9-a9ba-c98e8179358a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:36.520160 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-8a5d2be2-a6fc-43c9-a9ba-c98e8179358a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:36.525294 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 855/1683] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:08:36 2022] DELETE /identity/v3/projects/df8d43e298c5449f8f0e8c3db7c07a41 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:08:36.525294 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5d2be2-a6fc-43c9-a9ba-c98e8179358a admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:36.525294 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-8a5d2be2-a6fc-43c9-a9ba-c98e8179358a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:36.581952 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-e6bb08e0-4e7e-408b-bfd9-d22cc3bc1be3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee3cd32b940b4d73bf000e04a6fc44a3'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:36.589731 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 829/1684] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:08:36 2022] DELETE /identity/v3/users/ee3cd32b940b4d73bf000e04a6fc44a3 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:08:36.590660 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bb08e0-4e7e-408b-bfd9-d22cc3bc1be3 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:36.592277 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-e6bb08e0-4e7e-408b-bfd9-d22cc3bc1be3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:36.624261 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5d2be2-a6fc-43c9-a9ba-c98e8179358a admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:36.651812 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:36.682501 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bb08e0-4e7e-408b-bfd9-d22cc3bc1be3 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:36.688739 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f74307-4bdb-47ba-a260-09717ef32a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:36.689323 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f74307-4bdb-47ba-a260-09717ef32a90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:36.689672 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f74307-4bdb-47ba-a260-09717ef32a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:36.690121 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f74307-4bdb-47ba-a260-09717ef32a90 admin admin] PATH_INFO: `/v3/users/d06f9702a94e45cb88ddfceffadcdeda` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:36.691523 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f74307-4bdb-47ba-a260-09717ef32a90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d06f9702a94e45cb88ddfceffadcdeda)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:36.695023 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f74307-4bdb-47ba-a260-09717ef32a90 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:36.737587 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a32f512-62d6-488e-acc2-8de8625591ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:36.738484 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a32f512-62d6-488e-acc2-8de8625591ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:36.738705 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a32f512-62d6-488e-acc2-8de8625591ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:36.739157 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a32f512-62d6-488e-acc2-8de8625591ce admin admin] PATH_INFO: `/v3/projects/fb4d940eede741469b90aa1b4d0cd4b3/users/e7d758dd576a4c35bee4e3b1d6c24029/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:36.741122 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a32f512-62d6-488e-acc2-8de8625591ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb4d940eede741469b90aa1b4d0cd4b3, user_id=e7d758dd576a4c35bee4e3b1d6c24029, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:36.744379 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a32f512-62d6-488e-acc2-8de8625591ce admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:36.761859 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 856/1685] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:08:36 2022] PUT /identity/v3/projects/fb4d940eede741469b90aa1b4d0cd4b3/users/e7d758dd576a4c35bee4e3b1d6c24029/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:08:36.778481 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0320e6-6bd6-4a11-9936-cf2eb0ca7e70 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:36.779423 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0320e6-6bd6-4a11-9936-cf2eb0ca7e70 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:36.780540 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0320e6-6bd6-4a11-9936-cf2eb0ca7e70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:36.809903 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:36.813438 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-1e0320e6-6bd6-4a11-9936-cf2eb0ca7e70 None None] MFA Rules not processed for user `e7d758dd576a4c35bee4e3b1d6c24029`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:08:36.814680 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-1e0320e6-6bd6-4a11-9936-cf2eb0ca7e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:36.896237 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 857/1686] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:08:36 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:08:36.924779 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0320e6-6bd6-4a11-9936-cf2eb0ca7e70 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:36.926241 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-1e0320e6-6bd6-4a11-9936-cf2eb0ca7e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:36.962138 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-37f74307-4bdb-47ba-a260-09717ef32a90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd06f9702a94e45cb88ddfceffadcdeda'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:36.991000 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-37f74307-4bdb-47ba-a260-09717ef32a90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd06f9702a94e45cb88ddfceffadcdeda'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:37.024510 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-37f74307-4bdb-47ba-a260-09717ef32a90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd06f9702a94e45cb88ddfceffadcdeda'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:37.025671 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-37f74307-4bdb-47ba-a260-09717ef32a90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd06f9702a94e45cb88ddfceffadcdeda'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:37.036347 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0320e6-6bd6-4a11-9936-cf2eb0ca7e70 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:37.036347 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008512cf-76a6-4222-9532-1cf974e967ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab599d98b0624a2e89090312bcab713f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:37.036347 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008512cf-76a6-4222-9532-1cf974e967ef service cinder] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:37.037726 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008512cf-76a6-4222-9532-1cf974e967ef service cinder] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:37.037726 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008512cf-76a6-4222-9532-1cf974e967ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:37.043479 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-008512cf-76a6-4222-9532-1cf974e967ef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:37.070575 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008512cf-76a6-4222-9532-1cf974e967ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:37.075015 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008512cf-76a6-4222-9532-1cf974e967ef service cinder] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:37.103426 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 858/1687] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:08:36 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 03:08:37.118779 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-37f74307-4bdb-47ba-a260-09717ef32a90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd06f9702a94e45cb88ddfceffadcdeda'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:37.125997 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 830/1688] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:08:36 2022] DELETE /identity/v3/users/d06f9702a94e45cb88ddfceffadcdeda => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:08:37.212550 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008512cf-76a6-4222-9532-1cf974e967ef service cinder] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:37.214100 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-008512cf-76a6-4222-9532-1cf974e967ef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:37.300931 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008512cf-76a6-4222-9532-1cf974e967ef service cinder] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:37.335646 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f74307-4bdb-47ba-a260-09717ef32a90 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:37.337134 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-37f74307-4bdb-47ba-a260-09717ef32a90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:37.398927 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d10c350-1f3b-41e8-b523-f60580653320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:37.399807 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d10c350-1f3b-41e8-b523-f60580653320 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:37.400512 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d10c350-1f3b-41e8-b523-f60580653320 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:37.402471 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:37.403279 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d10c350-1f3b-41e8-b523-f60580653320 admin admin] PATH_INFO: `/v3/projects/83f9b422ab7b466e8b88aa0a4f2c04c3` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:37.410447 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d10c350-1f3b-41e8-b523-f60580653320 admin admin] RBAC: Authorizing `identity:delete_project(project_id=83f9b422ab7b466e8b88aa0a4f2c04c3)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:37.410447 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d10c350-1f3b-41e8-b523-f60580653320 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:37.410638 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f74307-4bdb-47ba-a260-09717ef32a90 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:37.415739 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a68fdb-d548-4587-bd40-04632695b682 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab599d98b0624a2e89090312bcab713f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:37.416365 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a68fdb-d548-4587-bd40-04632695b682 service cinder] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:37.416673 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a68fdb-d548-4587-bd40-04632695b682 service cinder] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:37.416961 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a68fdb-d548-4587-bd40-04632695b682 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:37.419142 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-e0a68fdb-d548-4587-bd40-04632695b682 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:37.519541 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a68fdb-d548-4587-bd40-04632695b682 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:37.519541 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a68fdb-d548-4587-bd40-04632695b682 service cinder] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:37.532635 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 831/1689] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:08:37 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 03:08:37.616099 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-9d10c350-1f3b-41e8-b523-f60580653320 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83f9b422ab7b466e8b88aa0a4f2c04c3', 'request_id': 'req-9d10c350-1f3b-41e8-b523-f60580653320'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:37.626826 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-9d10c350-1f3b-41e8-b523-f60580653320 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83f9b422ab7b466e8b88aa0a4f2c04c3', 'request_id': 'req-9d10c350-1f3b-41e8-b523-f60580653320'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:37.690574 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-9d10c350-1f3b-41e8-b523-f60580653320 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83f9b422ab7b466e8b88aa0a4f2c04c3', 'request_id': 'req-9d10c350-1f3b-41e8-b523-f60580653320'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:37.708552 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-9d10c350-1f3b-41e8-b523-f60580653320 admin admin] The token cache is being invalidate because project 83f9b422ab7b466e8b88aa0a4f2c04c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81517) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:08:37.709361 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-9d10c350-1f3b-41e8-b523-f60580653320 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:37.711431 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-9d10c350-1f3b-41e8-b523-f60580653320 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:37.719119 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 859/1690] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:08:37 2022] DELETE /identity/v3/projects/83f9b422ab7b466e8b88aa0a4f2c04c3 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:08:37.720147 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396be87e-cf12-4e43-9985-f418ba2b373f None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:37.720445 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396be87e-cf12-4e43-9985-f418ba2b373f None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:37.720724 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396be87e-cf12-4e43-9985-f418ba2b373f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:37.741319 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-396be87e-cf12-4e43-9985-f418ba2b373f None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:08:37.742645 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-396be87e-cf12-4e43-9985-f418ba2b373f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:37.748190 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:37.837001 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 832/1691] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:08:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:08:37.845158 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb4055d-ed7a-436e-8afa-3e25b92e577b None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:37.845665 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb4055d-ed7a-436e-8afa-3e25b92e577b None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:37.845742 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb4055d-ed7a-436e-8afa-3e25b92e577b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:37.866420 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-1eb4055d-ed7a-436e-8afa-3e25b92e577b None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:08:37.867636 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-1eb4055d-ed7a-436e-8afa-3e25b92e577b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:37.880429 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396be87e-cf12-4e43-9985-f418ba2b373f None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:37.883433 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 860/1692] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:08:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:08:37.884211 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-396be87e-cf12-4e43-9985-f418ba2b373f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:37.894179 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb4055d-ed7a-436e-8afa-3e25b92e577b None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:37.910577 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb4055d-ed7a-436e-8afa-3e25b92e577b None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:37.910787 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396be87e-cf12-4e43-9985-f418ba2b373f None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:37.915652 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699df643-c323-4b06-bf04-68672fef775b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:37.918079 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699df643-c323-4b06-bf04-68672fef775b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:37.918079 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699df643-c323-4b06-bf04-68672fef775b admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:37.918079 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699df643-c323-4b06-bf04-68672fef775b admin admin] PATH_INFO: `/v3/projects/56b6a1a123da472982f7a66a900015f1` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:37.918792 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699df643-c323-4b06-bf04-68672fef775b admin admin] RBAC: Authorizing `identity:delete_project(project_id=56b6a1a123da472982f7a66a900015f1)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:37.930489 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699df643-c323-4b06-bf04-68672fef775b admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:37.934703 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebef831-ed2b-424a-aa86-553425f3fedb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:37.935589 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebef831-ed2b-424a-aa86-553425f3fedb admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:37.941014 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebef831-ed2b-424a-aa86-553425f3fedb admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:37.941014 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebef831-ed2b-424a-aa86-553425f3fedb admin admin] PATH_INFO: `/v3/domains` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:37.941014 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebef831-ed2b-424a-aa86-553425f3fedb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 03:08:37.941014 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebef831-ed2b-424a-aa86-553425f3fedb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:37.941919 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebef831-ed2b-424a-aa86-553425f3fedb admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:37.965719 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 861/1693] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Oct 14 03:08:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 03:08:37.967046 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebef831-ed2b-424a-aa86-553425f3fedb admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:37.983155 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebef831-ed2b-424a-aa86-553425f3fedb admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:37.988596 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053f071b-621b-48a4-b594-0a3a04a2a767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:37.989211 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053f071b-621b-48a4-b594-0a3a04a2a767 admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:37.989498 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053f071b-621b-48a4-b594-0a3a04a2a767 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:37.989868 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053f071b-621b-48a4-b594-0a3a04a2a767 admin admin] PATH_INFO: `/v3/projects` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:37.990504 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053f071b-621b-48a4-b594-0a3a04a2a767 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:37.994095 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053f071b-621b-48a4-b594-0a3a04a2a767 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:38.069021 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-699df643-c323-4b06-bf04-68672fef775b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56b6a1a123da472982f7a66a900015f1', 'request_id': 'req-699df643-c323-4b06-bf04-68672fef775b'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:38.083925 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-699df643-c323-4b06-bf04-68672fef775b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56b6a1a123da472982f7a66a900015f1', 'request_id': 'req-699df643-c323-4b06-bf04-68672fef775b'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:38.100591 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 862/1694] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:08:37 2022] POST /identity/v3/projects => generated 383 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:08:38.101855 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053f071b-621b-48a4-b594-0a3a04a2a767 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:38.111319 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-053f071b-621b-48a4-b594-0a3a04a2a767 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:38.182546 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-699df643-c323-4b06-bf04-68672fef775b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56b6a1a123da472982f7a66a900015f1', 'request_id': 'req-699df643-c323-4b06-bf04-68672fef775b'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:38.198463 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053f071b-621b-48a4-b594-0a3a04a2a767 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:38.200046 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-699df643-c323-4b06-bf04-68672fef775b admin admin] The token cache is being invalidate because project 56b6a1a123da472982f7a66a900015f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:08:38.200480 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-699df643-c323-4b06-bf04-68672fef775b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:38.201241 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-699df643-c323-4b06-bf04-68672fef775b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:38.203444 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 833/1695] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:08:37 2022] DELETE /identity/v3/projects/56b6a1a123da472982f7a66a900015f1 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:08:38.204303 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699df643-c323-4b06-bf04-68672fef775b admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:38.213673 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-699df643-c323-4b06-bf04-68672fef775b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:38.213912 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28c39e0-e521-44fc-bcc2-dcc9e84bc08f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:38.213912 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28c39e0-e521-44fc-bcc2-dcc9e84bc08f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:38.213912 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28c39e0-e521-44fc-bcc2-dcc9e84bc08f admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:38.213912 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28c39e0-e521-44fc-bcc2-dcc9e84bc08f admin admin] PATH_INFO: `/v3/users/ce924de479c44547a27ea09a3de9e980` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:38.214226 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28c39e0-e521-44fc-bcc2-dcc9e84bc08f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce924de479c44547a27ea09a3de9e980)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:38.214226 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28c39e0-e521-44fc-bcc2-dcc9e84bc08f admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:38.291061 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699df643-c323-4b06-bf04-68672fef775b admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:38.295710 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5a4412-e0bc-4717-91a9-5a7c75e17e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:38.299754 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5a4412-e0bc-4717-91a9-5a7c75e17e1b admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:38.299754 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5a4412-e0bc-4717-91a9-5a7c75e17e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:38.299754 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5a4412-e0bc-4717-91a9-5a7c75e17e1b admin admin] PATH_INFO: `/v3/users` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:38.299754 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5a4412-e0bc-4717-91a9-5a7c75e17e1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:38.299754 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5a4412-e0bc-4717-91a9-5a7c75e17e1b admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:38.399331 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 834/1696] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:08:38 2022] POST /identity/v3/users => generated 292 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:08:38.400306 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5a4412-e0bc-4717-91a9-5a7c75e17e1b admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:38.401894 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-9b5a4412-e0bc-4717-91a9-5a7c75e17e1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:38.415976 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:38.416126 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5a4412-e0bc-4717-91a9-5a7c75e17e1b admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:38.421211 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dac6781-162c-4c6a-a487-6e606a514ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:38.421880 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dac6781-162c-4c6a-a487-6e606a514ce5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:38.422257 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dac6781-162c-4c6a-a487-6e606a514ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:38.422586 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dac6781-162c-4c6a-a487-6e606a514ce5 admin admin] PATH_INFO: `/v3/users/62f14f2eb3c9491cb86fe6142abb18e5` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:38.422708 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-a28c39e0-e521-44fc-bcc2-dcc9e84bc08f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce924de479c44547a27ea09a3de9e980'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:38.423341 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-a28c39e0-e521-44fc-bcc2-dcc9e84bc08f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce924de479c44547a27ea09a3de9e980'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:38.423772 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dac6781-162c-4c6a-a487-6e606a514ce5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=62f14f2eb3c9491cb86fe6142abb18e5)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:38.425834 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dac6781-162c-4c6a-a487-6e606a514ce5 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:38.444052 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=88,keystone:DELETE=8,keystone:INSERT=4 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:38.478654 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-a28c39e0-e521-44fc-bcc2-dcc9e84bc08f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce924de479c44547a27ea09a3de9e980'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:38.479285 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-a28c39e0-e521-44fc-bcc2-dcc9e84bc08f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce924de479c44547a27ea09a3de9e980'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:38.487204 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-a28c39e0-e521-44fc-bcc2-dcc9e84bc08f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce924de479c44547a27ea09a3de9e980'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:38.505106 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 863/1697] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:08:38 2022] DELETE /identity/v3/users/ce924de479c44547a27ea09a3de9e980 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:08:38.505956 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28c39e0-e521-44fc-bcc2-dcc9e84bc08f admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:38.508195 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-a28c39e0-e521-44fc-bcc2-dcc9e84bc08f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:38.571914 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28c39e0-e521-44fc-bcc2-dcc9e84bc08f admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:38.576485 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766a0fa6-c253-4773-8428-d417e15f9978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:38.577080 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766a0fa6-c253-4773-8428-d417e15f9978 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:38.577360 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766a0fa6-c253-4773-8428-d417e15f9978 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:38.577639 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766a0fa6-c253-4773-8428-d417e15f9978 admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:38.578300 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766a0fa6-c253-4773-8428-d417e15f9978 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:38.579966 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766a0fa6-c253-4773-8428-d417e15f9978 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:38.589603 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 864/1698] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:08:38 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:08:38.590791 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39476455-6636-4120-8ecb-74f77d5d0eaf None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:38.591078 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39476455-6636-4120-8ecb-74f77d5d0eaf None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:38.591408 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39476455-6636-4120-8ecb-74f77d5d0eaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:38.596773 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-5dac6781-162c-4c6a-a487-6e606a514ce5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62f14f2eb3c9491cb86fe6142abb18e5'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:38.605578 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-5dac6781-162c-4c6a-a487-6e606a514ce5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62f14f2eb3c9491cb86fe6142abb18e5'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:38.610419 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-39476455-6636-4120-8ecb-74f77d5d0eaf None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:08:38.611619 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-39476455-6636-4120-8ecb-74f77d5d0eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:38.618576 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-5dac6781-162c-4c6a-a487-6e606a514ce5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62f14f2eb3c9491cb86fe6142abb18e5'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:38.619283 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-5dac6781-162c-4c6a-a487-6e606a514ce5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62f14f2eb3c9491cb86fe6142abb18e5'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:38.620292 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 865/1699] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:08:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:08:38.621064 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39476455-6636-4120-8ecb-74f77d5d0eaf None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:38.630949 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39476455-6636-4120-8ecb-74f77d5d0eaf None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:38.635690 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818a21b5-562f-45a3-8da3-7193fd48bbfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:38.636286 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818a21b5-562f-45a3-8da3-7193fd48bbfe admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:38.636570 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818a21b5-562f-45a3-8da3-7193fd48bbfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:38.636845 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818a21b5-562f-45a3-8da3-7193fd48bbfe admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:38.637513 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818a21b5-562f-45a3-8da3-7193fd48bbfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:38.639152 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818a21b5-562f-45a3-8da3-7193fd48bbfe admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:38.652246 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 866/1700] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:08:38 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:08:38.652246 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818a21b5-562f-45a3-8da3-7193fd48bbfe admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:38.652246 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-818a21b5-562f-45a3-8da3-7193fd48bbfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:38.719060 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-5dac6781-162c-4c6a-a487-6e606a514ce5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62f14f2eb3c9491cb86fe6142abb18e5'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:38.725646 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 835/1701] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:08:38 2022] DELETE /identity/v3/users/62f14f2eb3c9491cb86fe6142abb18e5 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:08:38.726237 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dac6781-162c-4c6a-a487-6e606a514ce5 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:38.728078 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-5dac6781-162c-4c6a-a487-6e606a514ce5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:38.768895 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818a21b5-562f-45a3-8da3-7193fd48bbfe admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:38.803407 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dac6781-162c-4c6a-a487-6e606a514ce5 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:38.809556 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8649a9f9-8e5b-4fc5-b107-e5bca12708d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:38.811311 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8649a9f9-8e5b-4fc5-b107-e5bca12708d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:38.811616 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8649a9f9-8e5b-4fc5-b107-e5bca12708d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:38.811873 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8649a9f9-8e5b-4fc5-b107-e5bca12708d6 admin admin] PATH_INFO: `/v3/projects/9e3dfb4977504552a5480cd375ded7dd/users/ced832cf86304814b020de4ed3e0955f/roles/9bd0470b544240209a420f640c79c06c` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:38.828538 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19bed4e-243c-4772-a76f-a2e2664c10c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:38.829163 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19bed4e-243c-4772-a76f-a2e2664c10c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:38.829458 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19bed4e-243c-4772-a76f-a2e2664c10c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:38.829735 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19bed4e-243c-4772-a76f-a2e2664c10c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:38.831838 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-b19bed4e-243c-4772-a76f-a2e2664c10c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:38.832041 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8649a9f9-8e5b-4fc5-b107-e5bca12708d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e3dfb4977504552a5480cd375ded7dd, user_id=ced832cf86304814b020de4ed3e0955f, role_id=9bd0470b544240209a420f640c79c06c)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:38.834780 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8649a9f9-8e5b-4fc5-b107-e5bca12708d6 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:38.841594 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:38.845515 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19bed4e-243c-4772-a76f-a2e2664c10c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:38.847057 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19bed4e-243c-4772-a76f-a2e2664c10c6 service neutron] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:38.856730 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 867/1702] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:08:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:08:38.936824 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 836/1703] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:08:38 2022] PUT /identity/v3/projects/9e3dfb4977504552a5480cd375ded7dd/users/ced832cf86304814b020de4ed3e0955f/roles/9bd0470b544240209a420f640c79c06c => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:08:38.942165 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e3c490-2557-419f-ae16-62b9e5e65f78 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:38.942461 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e3c490-2557-419f-ae16-62b9e5e65f78 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:38.942740 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e3c490-2557-419f-ae16-62b9e5e65f78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:38.963635 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-e2e3c490-2557-419f-ae16-62b9e5e65f78 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:08:38.964854 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-e2e3c490-2557-419f-ae16-62b9e5e65f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:39.034945 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 868/1704] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:08:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:08:39.049195 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8649a9f9-8e5b-4fc5-b107-e5bca12708d6 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:39.050935 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-8649a9f9-8e5b-4fc5-b107-e5bca12708d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:39.123424 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8649a9f9-8e5b-4fc5-b107-e5bca12708d6 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:39.127525 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445986c3-62a4-4b5d-bdee-f72d170ef0c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:39.128224 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445986c3-62a4-4b5d-bdee-f72d170ef0c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:39.128702 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445986c3-62a4-4b5d-bdee-f72d170ef0c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:39.128974 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445986c3-62a4-4b5d-bdee-f72d170ef0c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:39.138784 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445986c3-62a4-4b5d-bdee-f72d170ef0c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:39.140281 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445986c3-62a4-4b5d-bdee-f72d170ef0c7 service neutron] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:39.150007 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 837/1705] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:08:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 03:08:39.218378 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e3c490-2557-419f-ae16-62b9e5e65f78 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:39.219860 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-e2e3c490-2557-419f-ae16-62b9e5e65f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:39.235907 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e3c490-2557-419f-ae16-62b9e5e65f78 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:39.242643 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a525339-d58c-4e2f-827b-b861e5963b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:39.245256 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a525339-d58c-4e2f-827b-b861e5963b4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:39.245645 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a525339-d58c-4e2f-827b-b861e5963b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:39.246067 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a525339-d58c-4e2f-827b-b861e5963b4f admin admin] PATH_INFO: `/v3/projects/ac9ea53b915248fa92bc5dfb5fc85785` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:39.247111 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a525339-d58c-4e2f-827b-b861e5963b4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac9ea53b915248fa92bc5dfb5fc85785)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:39.249173 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a525339-d58c-4e2f-827b-b861e5963b4f admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:39.375330 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445986c3-62a4-4b5d-bdee-f72d170ef0c7 service neutron] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:39.377112 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-445986c3-62a4-4b5d-bdee-f72d170ef0c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:39.387355 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-7a525339-d58c-4e2f-827b-b861e5963b4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac9ea53b915248fa92bc5dfb5fc85785', 'request_id': 'req-7a525339-d58c-4e2f-827b-b861e5963b4f'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:39.396822 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445986c3-62a4-4b5d-bdee-f72d170ef0c7 service neutron] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:39.398014 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-7a525339-d58c-4e2f-827b-b861e5963b4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac9ea53b915248fa92bc5dfb5fc85785', 'request_id': 'req-7a525339-d58c-4e2f-827b-b861e5963b4f'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:39.420340 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:39.477702 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f8acc1-a954-4415-80fa-249082367c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:39.478375 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f8acc1-a954-4415-80fa-249082367c2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:39.500029 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f8acc1-a954-4415-80fa-249082367c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:39.500029 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f8acc1-a954-4415-80fa-249082367c2b admin admin] PATH_INFO: `/v3/projects/ff08703801ed4e03a584956222704d4a` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:39.500029 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f8acc1-a954-4415-80fa-249082367c2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff08703801ed4e03a584956222704d4a)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:39.500029 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f8acc1-a954-4415-80fa-249082367c2b admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:39.519202 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-7a525339-d58c-4e2f-827b-b861e5963b4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac9ea53b915248fa92bc5dfb5fc85785', 'request_id': 'req-7a525339-d58c-4e2f-827b-b861e5963b4f'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:39.532720 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-7a525339-d58c-4e2f-827b-b861e5963b4f admin admin] The token cache is being invalidate because project ac9ea53b915248fa92bc5dfb5fc85785 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81517) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:08:39.533014 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-7a525339-d58c-4e2f-827b-b861e5963b4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:39.533672 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-7a525339-d58c-4e2f-827b-b861e5963b4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:39.535575 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 869/1706] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:08:39 2022] DELETE /identity/v3/projects/ac9ea53b915248fa92bc5dfb5fc85785 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:08:39.541999 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cd3dbb-70da-4b40-82db-e667c3ea061d None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:39.542288 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cd3dbb-70da-4b40-82db-e667c3ea061d None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:39.542564 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cd3dbb-70da-4b40-82db-e667c3ea061d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:39.562924 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-41cd3dbb-70da-4b40-82db-e667c3ea061d None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:08:39.564018 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-41cd3dbb-70da-4b40-82db-e667c3ea061d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:39.578060 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 870/1707] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:08:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 03:08:39.582102 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754d757f-ed77-4fa3-b54d-8bd309c8981b None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:39.586450 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754d757f-ed77-4fa3-b54d-8bd309c8981b None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:39.586450 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754d757f-ed77-4fa3-b54d-8bd309c8981b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:39.603328 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-754d757f-ed77-4fa3-b54d-8bd309c8981b None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:08:39.604528 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-754d757f-ed77-4fa3-b54d-8bd309c8981b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:39.619952 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 871/1708] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:08:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 03:08:39.619952 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754d757f-ed77-4fa3-b54d-8bd309c8981b None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:39.628224 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754d757f-ed77-4fa3-b54d-8bd309c8981b None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:39.629064 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-f7f8acc1-a954-4415-80fa-249082367c2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff08703801ed4e03a584956222704d4a', 'request_id': 'req-f7f8acc1-a954-4415-80fa-249082367c2b'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:39.636398 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-f7f8acc1-a954-4415-80fa-249082367c2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff08703801ed4e03a584956222704d4a', 'request_id': 'req-f7f8acc1-a954-4415-80fa-249082367c2b'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:39.688325 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c370a77-a068-43af-8ad5-f4df4a3c3944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:39.688969 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c370a77-a068-43af-8ad5-f4df4a3c3944 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:39.689234 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c370a77-a068-43af-8ad5-f4df4a3c3944 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:39.689503 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c370a77-a068-43af-8ad5-f4df4a3c3944 admin admin] PATH_INFO: `/v3/domains` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:39.690178 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c370a77-a068-43af-8ad5-f4df4a3c3944 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 03:08:39.690482 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c370a77-a068-43af-8ad5-f4df4a3c3944 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:39.692083 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c370a77-a068-43af-8ad5-f4df4a3c3944 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:39.698540 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:39.703427 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 872/1709] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Oct 14 03:08:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 03:08:39.707919 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c370a77-a068-43af-8ad5-f4df4a3c3944 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:39.718058 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c370a77-a068-43af-8ad5-f4df4a3c3944 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:39.722889 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45159d8e-beb4-476f-871f-b5cb64ed00bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:39.723476 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45159d8e-beb4-476f-871f-b5cb64ed00bf admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:39.723756 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45159d8e-beb4-476f-871f-b5cb64ed00bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:39.724027 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45159d8e-beb4-476f-871f-b5cb64ed00bf admin admin] PATH_INFO: `/v3/projects` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:39.724728 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45159d8e-beb4-476f-871f-b5cb64ed00bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:39.727321 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45159d8e-beb4-476f-871f-b5cb64ed00bf admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:39.735907 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-f7f8acc1-a954-4415-80fa-249082367c2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff08703801ed4e03a584956222704d4a', 'request_id': 'req-f7f8acc1-a954-4415-80fa-249082367c2b'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:39.748353 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-f7f8acc1-a954-4415-80fa-249082367c2b admin admin] The token cache is being invalidate because project ff08703801ed4e03a584956222704d4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:08:39.748641 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-f7f8acc1-a954-4415-80fa-249082367c2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:39.749280 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-f7f8acc1-a954-4415-80fa-249082367c2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:39.750808 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 838/1710] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:08:39 2022] DELETE /identity/v3/projects/ff08703801ed4e03a584956222704d4a => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 03:08:39.754790 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f8acc1-a954-4415-80fa-249082367c2b admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:39.756185 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-f7f8acc1-a954-4415-80fa-249082367c2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:39.768498 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f8acc1-a954-4415-80fa-249082367c2b admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:39.775090 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abda185-1db9-4d5d-af40-ad06c64e3bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:39.775718 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abda185-1db9-4d5d-af40-ad06c64e3bfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:39.775997 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abda185-1db9-4d5d-af40-ad06c64e3bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:39.776269 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abda185-1db9-4d5d-af40-ad06c64e3bfd admin admin] PATH_INFO: `/v3/users/e7d758dd576a4c35bee4e3b1d6c24029` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:39.777561 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abda185-1db9-4d5d-af40-ad06c64e3bfd admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7d758dd576a4c35bee4e3b1d6c24029)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:39.780202 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abda185-1db9-4d5d-af40-ad06c64e3bfd admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:39.838760 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 873/1711] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:08:39 2022] POST /identity/v3/projects => generated 389 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 03:08:39.843419 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45159d8e-beb4-476f-871f-b5cb64ed00bf admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:39.846002 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-45159d8e-beb4-476f-871f-b5cb64ed00bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:39.917028 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45159d8e-beb4-476f-871f-b5cb64ed00bf admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:39.921934 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb361aef-d29b-47c0-9609-bfa1f4166b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:39.922549 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb361aef-d29b-47c0-9609-bfa1f4166b97 admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:39.922861 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb361aef-d29b-47c0-9609-bfa1f4166b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:39.923107 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb361aef-d29b-47c0-9609-bfa1f4166b97 admin admin] PATH_INFO: `/v3/users` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:39.923746 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb361aef-d29b-47c0-9609-bfa1f4166b97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:39.925704 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb361aef-d29b-47c0-9609-bfa1f4166b97 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:39.995907 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-2abda185-1db9-4d5d-af40-ad06c64e3bfd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7d758dd576a4c35bee4e3b1d6c24029'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:40.003602 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-2abda185-1db9-4d5d-af40-ad06c64e3bfd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7d758dd576a4c35bee4e3b1d6c24029'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:40.018128 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-2abda185-1db9-4d5d-af40-ad06c64e3bfd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7d758dd576a4c35bee4e3b1d6c24029'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:40.018763 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-2abda185-1db9-4d5d-af40-ad06c64e3bfd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7d758dd576a4c35bee4e3b1d6c24029'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:40.023527 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 874/1712] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:08:39 2022] POST /identity/v3/users => generated 295 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 03:08:40.027583 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb361aef-d29b-47c0-9609-bfa1f4166b97 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:40.029375 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-eb361aef-d29b-47c0-9609-bfa1f4166b97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:40.041699 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb361aef-d29b-47c0-9609-bfa1f4166b97 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:40.046761 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e04854f-0982-4d0b-bad8-7e31f61c02b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:40.047372 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e04854f-0982-4d0b-bad8-7e31f61c02b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:40.048036 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e04854f-0982-4d0b-bad8-7e31f61c02b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:40.048036 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e04854f-0982-4d0b-bad8-7e31f61c02b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:40.048551 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e04854f-0982-4d0b-bad8-7e31f61c02b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:40.050200 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e04854f-0982-4d0b-bad8-7e31f61c02b8 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:40.060531 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 875/1713] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:08:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:08:40.065350 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e04854f-0982-4d0b-bad8-7e31f61c02b8 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:40.077618 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e04854f-0982-4d0b-bad8-7e31f61c02b8 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:40.082329 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8af3a7a-49da-46f4-a67a-032f2f640e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:40.082935 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8af3a7a-49da-46f4-a67a-032f2f640e3f admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:40.083214 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8af3a7a-49da-46f4-a67a-032f2f640e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:40.083492 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8af3a7a-49da-46f4-a67a-032f2f640e3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:40.084137 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8af3a7a-49da-46f4-a67a-032f2f640e3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:40.086328 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8af3a7a-49da-46f4-a67a-032f2f640e3f admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:40.100174 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 876/1714] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:08:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:08:40.101237 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8af3a7a-49da-46f4-a67a-032f2f640e3f admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:40.111373 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8af3a7a-49da-46f4-a67a-032f2f640e3f admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:40.116033 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64922d5-bc88-46f4-8bfd-42328e4049f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:40.116721 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64922d5-bc88-46f4-8bfd-42328e4049f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:40.116999 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64922d5-bc88-46f4-8bfd-42328e4049f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:40.117270 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64922d5-bc88-46f4-8bfd-42328e4049f7 admin admin] PATH_INFO: `/v3/projects/d3229255075a428b83187644d2f1602e/users/09872aa13d324a02a4553745d67d030f/roles/9bd0470b544240209a420f640c79c06c` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:40.124450 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-2abda185-1db9-4d5d-af40-ad06c64e3bfd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7d758dd576a4c35bee4e3b1d6c24029'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:40.130962 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 839/1715] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:08:39 2022] DELETE /identity/v3/users/e7d758dd576a4c35bee4e3b1d6c24029 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 03:08:40.132576 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64922d5-bc88-46f4-8bfd-42328e4049f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3229255075a428b83187644d2f1602e, user_id=09872aa13d324a02a4553745d67d030f, role_id=9bd0470b544240209a420f640c79c06c)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:40.135373 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64922d5-bc88-46f4-8bfd-42328e4049f7 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:40.218660 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 877/1716] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:08:40 2022] PUT /identity/v3/projects/d3229255075a428b83187644d2f1602e/users/09872aa13d324a02a4553745d67d030f/roles/9bd0470b544240209a420f640c79c06c => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 03:08:40.230008 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9410bc0b-0209-4c42-b7e0-091148e7c624 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:40.230299 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9410bc0b-0209-4c42-b7e0-091148e7c624 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:40.230636 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9410bc0b-0209-4c42-b7e0-091148e7c624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:40.261673 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-9410bc0b-0209-4c42-b7e0-091148e7c624 None None] MFA Rules not processed for user `09872aa13d324a02a4553745d67d030f`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:08:40.262992 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-9410bc0b-0209-4c42-b7e0-091148e7c624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:40.272729 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:40.329227 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 840/1717] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:08:40 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:08:40.333930 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64922d5-bc88-46f4-8bfd-42328e4049f7 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:40.395363 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64922d5-bc88-46f4-8bfd-42328e4049f7 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:40.400179 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e5df92-02f7-4a3b-bb16-cd749ed5dd73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:40.400783 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e5df92-02f7-4a3b-bb16-cd749ed5dd73 admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:40.401188 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e5df92-02f7-4a3b-bb16-cd749ed5dd73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:40.401472 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e5df92-02f7-4a3b-bb16-cd749ed5dd73 admin admin] PATH_INFO: `/v3/projects` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:40.402280 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e5df92-02f7-4a3b-bb16-cd749ed5dd73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:40.404189 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e5df92-02f7-4a3b-bb16-cd749ed5dd73 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:40.545055 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 878/1718] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:08:40 2022] POST /identity/v3/projects => generated 387 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:08:40.556061 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9410bc0b-0209-4c42-b7e0-091148e7c624 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:40.568785 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9410bc0b-0209-4c42-b7e0-091148e7c624 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:40.645817 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e826718f-5b56-45e9-a59f-ddf80a40630a None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:40.646180 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e826718f-5b56-45e9-a59f-ddf80a40630a None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:40.646527 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e826718f-5b56-45e9-a59f-ddf80a40630a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:40.649534 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb52054-2261-4ebe-8ea7-7373b7089739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:40.650308 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb52054-2261-4ebe-8ea7-7373b7089739 admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:40.650662 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb52054-2261-4ebe-8ea7-7373b7089739 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:40.651013 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb52054-2261-4ebe-8ea7-7373b7089739 admin admin] PATH_INFO: `/v3/users` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:40.652570 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb52054-2261-4ebe-8ea7-7373b7089739 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:40.654982 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb52054-2261-4ebe-8ea7-7373b7089739 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:40.656264 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:40.686573 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-e826718f-5b56-45e9-a59f-ddf80a40630a None None] MFA Rules not processed for user `ced832cf86304814b020de4ed3e0955f`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:08:40.688213 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-e826718f-5b56-45e9-a59f-ddf80a40630a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:40.753352 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 841/1719] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:08:40 2022] POST /identity/v3/users => generated 300 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:08:40.754464 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb52054-2261-4ebe-8ea7-7373b7089739 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:40.758117 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-2fb52054-2261-4ebe-8ea7-7373b7089739 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:40.770624 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb52054-2261-4ebe-8ea7-7373b7089739 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:40.775666 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6173ec18-76f3-4d75-b7c9-f58e132b6c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:40.776400 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6173ec18-76f3-4d75-b7c9-f58e132b6c6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:40.777134 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6173ec18-76f3-4d75-b7c9-f58e132b6c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:40.777134 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6173ec18-76f3-4d75-b7c9-f58e132b6c6a admin admin] PATH_INFO: `/v3/projects/fb4d940eede741469b90aa1b4d0cd4b3` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:40.777494 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 879/1720] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:08:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 03:08:40.781126 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e826718f-5b56-45e9-a59f-ddf80a40630a None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:40.782223 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6173ec18-76f3-4d75-b7c9-f58e132b6c6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb4d940eede741469b90aa1b4d0cd4b3)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:40.784142 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6173ec18-76f3-4d75-b7c9-f58e132b6c6a admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:40.790942 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e826718f-5b56-45e9-a59f-ddf80a40630a None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:40.795816 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa0bc27-4c96-4078-8e87-069eca31e10f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:40.796418 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa0bc27-4c96-4078-8e87-069eca31e10f admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:40.796716 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa0bc27-4c96-4078-8e87-069eca31e10f admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:40.797125 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa0bc27-4c96-4078-8e87-069eca31e10f admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:40.797834 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa0bc27-4c96-4078-8e87-069eca31e10f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:40.800081 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa0bc27-4c96-4078-8e87-069eca31e10f admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:40.811015 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 880/1721] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:08:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:08:40.812097 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa0bc27-4c96-4078-8e87-069eca31e10f admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:40.814216 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-2fa0bc27-4c96-4078-8e87-069eca31e10f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:40.871033 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa0bc27-4c96-4078-8e87-069eca31e10f admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:40.874718 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3523d9c4-9282-4c09-999a-a63b608a4046 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab599d98b0624a2e89090312bcab713f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:40.875317 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3523d9c4-9282-4c09-999a-a63b608a4046 service cinder] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:40.875716 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3523d9c4-9282-4c09-999a-a63b608a4046 service cinder] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:40.875998 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3523d9c4-9282-4c09-999a-a63b608a4046 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:40.884359 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3523d9c4-9282-4c09-999a-a63b608a4046 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:40.885814 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3523d9c4-9282-4c09-999a-a63b608a4046 service cinder] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:40.894796 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 881/1722] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:08:40 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:08:40.895650 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3523d9c4-9282-4c09-999a-a63b608a4046 service cinder] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:40.905598 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3523d9c4-9282-4c09-999a-a63b608a4046 service cinder] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:40.907114 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-6173ec18-76f3-4d75-b7c9-f58e132b6c6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb4d940eede741469b90aa1b4d0cd4b3', 'request_id': 'req-6173ec18-76f3-4d75-b7c9-f58e132b6c6a'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:40.916999 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-6173ec18-76f3-4d75-b7c9-f58e132b6c6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb4d940eede741469b90aa1b4d0cd4b3', 'request_id': 'req-6173ec18-76f3-4d75-b7c9-f58e132b6c6a'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:40.964674 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a12d49-163a-4234-90b6-f18123719f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:40.965400 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a12d49-163a-4234-90b6-f18123719f5c admin admin] REQUEST_METHOD: `PUT` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:40.965687 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a12d49-163a-4234-90b6-f18123719f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:40.966012 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a12d49-163a-4234-90b6-f18123719f5c admin admin] PATH_INFO: `/v3/projects/2a1efcc350544e2da5c8fd73ac91e55b/users/cf807f947fb0487eb8b45e44a206d14f/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:40.970505 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-6173ec18-76f3-4d75-b7c9-f58e132b6c6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb4d940eede741469b90aa1b4d0cd4b3', 'request_id': 'req-6173ec18-76f3-4d75-b7c9-f58e132b6c6a'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:40.981633 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a12d49-163a-4234-90b6-f18123719f5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a1efcc350544e2da5c8fd73ac91e55b, user_id=cf807f947fb0487eb8b45e44a206d14f, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:40.982997 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-6173ec18-76f3-4d75-b7c9-f58e132b6c6a admin admin] The token cache is being invalidate because project fb4d940eede741469b90aa1b4d0cd4b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:08:40.983284 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-6173ec18-76f3-4d75-b7c9-f58e132b6c6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:40.984187 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-6173ec18-76f3-4d75-b7c9-f58e132b6c6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:40.984976 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a12d49-163a-4234-90b6-f18123719f5c admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:40.985698 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 842/1723] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:08:40 2022] DELETE /identity/v3/projects/fb4d940eede741469b90aa1b4d0cd4b3 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:08:40.990505 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40310c1-776a-441d-9f34-93eda0f29211 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:40.990788 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40310c1-776a-441d-9f34-93eda0f29211 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:40.991084 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40310c1-776a-441d-9f34-93eda0f29211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:41.019941 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-c40310c1-776a-441d-9f34-93eda0f29211 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:08:41.021236 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-c40310c1-776a-441d-9f34-93eda0f29211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:41.030212 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 843/1724] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:08:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 03:08:41.037777 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f5413-f551-46d3-82dc-2ac62a7507cb None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:41.038608 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f5413-f551-46d3-82dc-2ac62a7507cb None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:41.038886 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f5413-f551-46d3-82dc-2ac62a7507cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:41.059170 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-b08f5413-f551-46d3-82dc-2ac62a7507cb None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:08:41.060432 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-b08f5413-f551-46d3-82dc-2ac62a7507cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:41.065995 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 882/1725] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:08:40 2022] PUT /identity/v3/projects/2a1efcc350544e2da5c8fd73ac91e55b/users/cf807f947fb0487eb8b45e44a206d14f/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:08:41.065995 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a12d49-163a-4234-90b6-f18123719f5c admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:41.067218 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-43a12d49-163a-4234-90b6-f18123719f5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:41.121031 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 844/1726] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:08:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 03:08:41.124978 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08f5413-f551-46d3-82dc-2ac62a7507cb None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:41.132317 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a12d49-163a-4234-90b6-f18123719f5c admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:41.133107 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:41.137598 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08f5413-f551-46d3-82dc-2ac62a7507cb None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:41.139417 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a976625-3076-4ca3-9431-4013409edc40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:41.140519 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a976625-3076-4ca3-9431-4013409edc40 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:41.140519 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a976625-3076-4ca3-9431-4013409edc40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:41.140778 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a976625-3076-4ca3-9431-4013409edc40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:41.141430 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a976625-3076-4ca3-9431-4013409edc40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:41.142408 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84638357-1d02-401a-8027-26d3a5d50fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:41.143003 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84638357-1d02-401a-8027-26d3a5d50fc8 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:41.143118 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a976625-3076-4ca3-9431-4013409edc40 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:41.143283 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84638357-1d02-401a-8027-26d3a5d50fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:41.143555 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84638357-1d02-401a-8027-26d3a5d50fc8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:41.144350 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84638357-1d02-401a-8027-26d3a5d50fc8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 03:08:41.144494 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84638357-1d02-401a-8027-26d3a5d50fc8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:41.146115 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84638357-1d02-401a-8027-26d3a5d50fc8 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:41.153713 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 883/1727] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:08:41 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 03:08:41.157691 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a976625-3076-4ca3-9431-4013409edc40 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:41.159677 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 845/1728] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Oct 14 03:08:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 03:08:41.166691 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84638357-1d02-401a-8027-26d3a5d50fc8 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:41.169322 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a976625-3076-4ca3-9431-4013409edc40 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:41.173958 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81834fc9-8429-4ee4-b7c1-936c9f11aa39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:41.174602 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81834fc9-8429-4ee4-b7c1-936c9f11aa39 admin admin] REQUEST_METHOD: `PUT` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:41.174884 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81834fc9-8429-4ee4-b7c1-936c9f11aa39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:41.175282 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81834fc9-8429-4ee4-b7c1-936c9f11aa39 admin admin] PATH_INFO: `/v3/projects/2a1efcc350544e2da5c8fd73ac91e55b/users/cf807f947fb0487eb8b45e44a206d14f/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:41.176125 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84638357-1d02-401a-8027-26d3a5d50fc8 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:41.177202 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81834fc9-8429-4ee4-b7c1-936c9f11aa39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a1efcc350544e2da5c8fd73ac91e55b, user_id=cf807f947fb0487eb8b45e44a206d14f, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:41.179951 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81834fc9-8429-4ee4-b7c1-936c9f11aa39 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:41.180663 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a5786b-589b-4c17-a4e8-ee75af38bd4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:41.181239 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a5786b-589b-4c17-a4e8-ee75af38bd4e admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:41.181519 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a5786b-589b-4c17-a4e8-ee75af38bd4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:41.181819 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a5786b-589b-4c17-a4e8-ee75af38bd4e admin admin] PATH_INFO: `/v3/projects` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:41.182446 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a5786b-589b-4c17-a4e8-ee75af38bd4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:41.184306 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a5786b-589b-4c17-a4e8-ee75af38bd4e admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:41.190344 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 884/1729] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:08:41 2022] PUT /identity/v3/projects/2a1efcc350544e2da5c8fd73ac91e55b/users/cf807f947fb0487eb8b45e44a206d14f/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 03:08:41.201359 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eddf315-787f-4ab4-83bf-e6c59279891e None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:41.201629 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eddf315-787f-4ab4-83bf-e6c59279891e None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:41.202108 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eddf315-787f-4ab4-83bf-e6c59279891e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:41.231092 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-5eddf315-787f-4ab4-83bf-e6c59279891e None None] MFA Rules not processed for user `cf807f947fb0487eb8b45e44a206d14f`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:08:41.232241 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-5eddf315-787f-4ab4-83bf-e6c59279891e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:41.273997 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 846/1730] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:08:41 2022] POST /identity/v3/projects => generated 375 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 03:08:41.278095 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a5786b-589b-4c17-a4e8-ee75af38bd4e admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:41.303165 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 885/1731] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:08:41 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 03:08:41.314170 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eddf315-787f-4ab4-83bf-e6c59279891e None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:41.315681 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-5eddf315-787f-4ab4-83bf-e6c59279891e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:41.329601 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:41.344311 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a5786b-589b-4c17-a4e8-ee75af38bd4e admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:41.349097 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b971b4c7-71d2-4e52-9962-295b0c73f485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:41.349705 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b971b4c7-71d2-4e52-9962-295b0c73f485 admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:41.350857 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b971b4c7-71d2-4e52-9962-295b0c73f485 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:41.351153 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b971b4c7-71d2-4e52-9962-295b0c73f485 admin admin] PATH_INFO: `/v3/users` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:41.351801 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b971b4c7-71d2-4e52-9962-295b0c73f485 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:41.353720 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b971b4c7-71d2-4e52-9962-295b0c73f485 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:41.378890 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eddf315-787f-4ab4-83bf-e6c59279891e None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:41.432312 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efca382f-e495-470f-b756-bca153f9d9c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab599d98b0624a2e89090312bcab713f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:41.436245 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efca382f-e495-470f-b756-bca153f9d9c5 service cinder] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:41.436245 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efca382f-e495-470f-b756-bca153f9d9c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:41.436245 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efca382f-e495-470f-b756-bca153f9d9c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:41.662125 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efca382f-e495-470f-b756-bca153f9d9c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:41.662125 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efca382f-e495-470f-b756-bca153f9d9c5 service cinder] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:41.662125 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 886/1732] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:08:41 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 03:08:41.714904 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 847/1733] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:08:41 2022] POST /identity/v3/users => generated 288 bytes in 436 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 03:08:41.727150 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efca382f-e495-470f-b756-bca153f9d9c5 service cinder] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:41.739660 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efca382f-e495-470f-b756-bca153f9d9c5 service cinder] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:41.745254 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c89d483-1c40-430c-be39-87003a1cea30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:41.746020 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c89d483-1c40-430c-be39-87003a1cea30 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:41.746374 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c89d483-1c40-430c-be39-87003a1cea30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:41.746709 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c89d483-1c40-430c-be39-87003a1cea30 admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:41.747477 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c89d483-1c40-430c-be39-87003a1cea30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:41.749116 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c89d483-1c40-430c-be39-87003a1cea30 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:41.764857 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 887/1734] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:08:41 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:08:41.766261 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b971b4c7-71d2-4e52-9962-295b0c73f485 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:41.783062 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b971b4c7-71d2-4e52-9962-295b0c73f485 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:41.783062 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718f3f73-7049-4d9f-8235-6d5da48fc513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:41.783062 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718f3f73-7049-4d9f-8235-6d5da48fc513 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:41.783062 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718f3f73-7049-4d9f-8235-6d5da48fc513 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:41.783645 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718f3f73-7049-4d9f-8235-6d5da48fc513 admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:41.783857 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718f3f73-7049-4d9f-8235-6d5da48fc513 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:41.785623 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718f3f73-7049-4d9f-8235-6d5da48fc513 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:41.786985 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c89d483-1c40-430c-be39-87003a1cea30 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:41.801025 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 848/1735] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:08:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:08:41.801223 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c89d483-1c40-430c-be39-87003a1cea30 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:41.802694 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718f3f73-7049-4d9f-8235-6d5da48fc513 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:41.807091 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7858ac-5186-4106-ae85-fe3e9c0f0577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:41.807980 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7858ac-5186-4106-ae85-fe3e9c0f0577 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:41.808264 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7858ac-5186-4106-ae85-fe3e9c0f0577 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:41.808540 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7858ac-5186-4106-ae85-fe3e9c0f0577 admin admin] PATH_INFO: `/v3/users/09872aa13d324a02a4553745d67d030f` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:41.809944 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7858ac-5186-4106-ae85-fe3e9c0f0577 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09872aa13d324a02a4553745d67d030f)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:41.812101 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7858ac-5186-4106-ae85-fe3e9c0f0577 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:41.814160 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718f3f73-7049-4d9f-8235-6d5da48fc513 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:41.831957 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f409d93-ff0c-4909-b6a0-0f3d69b9eed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:41.831957 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f409d93-ff0c-4909-b6a0-0f3d69b9eed6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:41.831957 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f409d93-ff0c-4909-b6a0-0f3d69b9eed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:41.831957 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f409d93-ff0c-4909-b6a0-0f3d69b9eed6 admin admin] PATH_INFO: `/v3/projects/b7e8f1717dd94efa843abbb993735c84/users/8e1de3646bfa4231ad5d3ed57a9fd028/roles/9bd0470b544240209a420f640c79c06c` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:41.837999 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f409d93-ff0c-4909-b6a0-0f3d69b9eed6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7e8f1717dd94efa843abbb993735c84, user_id=8e1de3646bfa4231ad5d3ed57a9fd028, role_id=9bd0470b544240209a420f640c79c06c)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:41.841161 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f409d93-ff0c-4909-b6a0-0f3d69b9eed6 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:42.075333 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 849/1736] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:08:41 2022] PUT /identity/v3/projects/b7e8f1717dd94efa843abbb993735c84/users/8e1de3646bfa4231ad5d3ed57a9fd028/roles/9bd0470b544240209a420f640c79c06c => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:08:42.086852 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5fd841-6a36-4442-92d1-d38558629f50 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:42.087158 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5fd841-6a36-4442-92d1-d38558629f50 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:42.087482 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5fd841-6a36-4442-92d1-d38558629f50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:42.117876 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-4e5fd841-6a36-4442-92d1-d38558629f50 None None] MFA Rules not processed for user `8e1de3646bfa4231ad5d3ed57a9fd028`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:08:42.119088 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-4e5fd841-6a36-4442-92d1-d38558629f50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:42.144871 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-7b7858ac-5186-4106-ae85-fe3e9c0f0577 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09872aa13d324a02a4553745d67d030f'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:42.145646 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-7b7858ac-5186-4106-ae85-fe3e9c0f0577 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09872aa13d324a02a4553745d67d030f'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:42.183495 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 850/1737] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:08:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 03:08:42.187940 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5fd841-6a36-4442-92d1-d38558629f50 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:42.189227 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-4e5fd841-6a36-4442-92d1-d38558629f50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:42.239564 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-7b7858ac-5186-4106-ae85-fe3e9c0f0577 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09872aa13d324a02a4553745d67d030f'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:42.240461 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-7b7858ac-5186-4106-ae85-fe3e9c0f0577 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09872aa13d324a02a4553745d67d030f'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:42.249103 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-7b7858ac-5186-4106-ae85-fe3e9c0f0577 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09872aa13d324a02a4553745d67d030f'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:42.253379 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5fd841-6a36-4442-92d1-d38558629f50 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:42.258172 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b02e773-8ab5-4de3-a86a-e9cc78214168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:42.258786 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b02e773-8ab5-4de3-a86a-e9cc78214168 admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:42.259071 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b02e773-8ab5-4de3-a86a-e9cc78214168 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:42.259344 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b02e773-8ab5-4de3-a86a-e9cc78214168 admin admin] PATH_INFO: `/v3/projects` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:42.260010 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b02e773-8ab5-4de3-a86a-e9cc78214168 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:42.261986 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b02e773-8ab5-4de3-a86a-e9cc78214168 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:42.270653 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 888/1738] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:08:41 2022] DELETE /identity/v3/users/09872aa13d324a02a4553745d67d030f => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 03:08:42.272636 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59339d7d-efec-49db-97fd-cbf213e3799a None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:42.272926 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59339d7d-efec-49db-97fd-cbf213e3799a None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:42.273201 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59339d7d-efec-49db-97fd-cbf213e3799a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:42.292450 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-59339d7d-efec-49db-97fd-cbf213e3799a None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:08:42.295902 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-59339d7d-efec-49db-97fd-cbf213e3799a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:42.345479 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 851/1739] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:08:42 2022] POST /identity/v3/projects => generated 373 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 03:08:42.347404 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b02e773-8ab5-4de3-a86a-e9cc78214168 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:42.362637 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 889/1740] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:08:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 03:08:42.373086 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59339d7d-efec-49db-97fd-cbf213e3799a None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:42.374646 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-59339d7d-efec-49db-97fd-cbf213e3799a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:42.408929 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b02e773-8ab5-4de3-a86a-e9cc78214168 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:42.416138 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68500bc1-80af-4a27-b352-e5bca2bbea5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:42.420156 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68500bc1-80af-4a27-b352-e5bca2bbea5f admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:42.420156 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68500bc1-80af-4a27-b352-e5bca2bbea5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:42.420156 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68500bc1-80af-4a27-b352-e5bca2bbea5f admin admin] PATH_INFO: `/v3/users` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:42.420156 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68500bc1-80af-4a27-b352-e5bca2bbea5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:42.421043 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68500bc1-80af-4a27-b352-e5bca2bbea5f admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:42.434396 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59339d7d-efec-49db-97fd-cbf213e3799a None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:42.482131 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682ed11e-0fd6-4fad-a57e-6e55a392e58f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:42.483197 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ed11e-0fd6-4fad-a57e-6e55a392e58f service neutron] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:42.483197 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ed11e-0fd6-4fad-a57e-6e55a392e58f service neutron] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:42.483510 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ed11e-0fd6-4fad-a57e-6e55a392e58f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:42.492896 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682ed11e-0fd6-4fad-a57e-6e55a392e58f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:42.494434 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682ed11e-0fd6-4fad-a57e-6e55a392e58f service neutron] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:42.503891 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 890/1741] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:08:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 03:08:42.539619 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 852/1742] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:08:42 2022] POST /identity/v3/users => generated 293 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:08:42.541701 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682ed11e-0fd6-4fad-a57e-6e55a392e58f service neutron] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:42.552247 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682ed11e-0fd6-4fad-a57e-6e55a392e58f service neutron] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:42.557774 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c51444-d287-4bc1-8f4c-ec6767e7ed32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:42.563591 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c51444-d287-4bc1-8f4c-ec6767e7ed32 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:42.563591 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c51444-d287-4bc1-8f4c-ec6767e7ed32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:42.563591 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c51444-d287-4bc1-8f4c-ec6767e7ed32 admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:42.563591 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c51444-d287-4bc1-8f4c-ec6767e7ed32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:42.563591 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c51444-d287-4bc1-8f4c-ec6767e7ed32 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:42.576422 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 891/1743] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:08:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:08:42.578276 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68500bc1-80af-4a27-b352-e5bca2bbea5f admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:42.589281 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68500bc1-80af-4a27-b352-e5bca2bbea5f admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:42.594155 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50837d72-0ad7-4667-b624-cc69de8624e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:42.594873 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50837d72-0ad7-4667-b624-cc69de8624e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:42.595158 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50837d72-0ad7-4667-b624-cc69de8624e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:42.595433 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50837d72-0ad7-4667-b624-cc69de8624e4 admin admin] PATH_INFO: `/v3/projects/d8dfedacd3b54f169972526b6a805b0d/users/20680a90b1c94164891d986feedc2b44/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:42.610828 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:42.611734 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50837d72-0ad7-4667-b624-cc69de8624e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8dfedacd3b54f169972526b6a805b0d, user_id=20680a90b1c94164891d986feedc2b44, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:42.614440 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50837d72-0ad7-4667-b624-cc69de8624e4 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:42.772729 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 853/1744] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:08:42 2022] PUT /identity/v3/projects/d8dfedacd3b54f169972526b6a805b0d/users/20680a90b1c94164891d986feedc2b44/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:08:42.777040 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c51444-d287-4bc1-8f4c-ec6767e7ed32 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:42.796219 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c51444-d287-4bc1-8f4c-ec6767e7ed32 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:42.857468 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a877af9b-a6b2-4e9c-b7f3-a3af1e519ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:42.858214 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a877af9b-a6b2-4e9c-b7f3-a3af1e519ca3 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:42.858494 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a877af9b-a6b2-4e9c-b7f3-a3af1e519ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:42.859044 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a877af9b-a6b2-4e9c-b7f3-a3af1e519ca3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:42.859686 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a877af9b-a6b2-4e9c-b7f3-a3af1e519ca3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:42.861337 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a877af9b-a6b2-4e9c-b7f3-a3af1e519ca3 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:42.871851 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 892/1745] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:08:42 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:08:42.876918 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50837d72-0ad7-4667-b624-cc69de8624e4 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:42.886783 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50837d72-0ad7-4667-b624-cc69de8624e4 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:42.891680 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa9a332-1a06-4963-829a-61f293df8a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:42.892661 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa9a332-1a06-4963-829a-61f293df8a42 admin admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:42.892661 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa9a332-1a06-4963-829a-61f293df8a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:42.892661 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa9a332-1a06-4963-829a-61f293df8a42 admin admin] PATH_INFO: `/v3/projects/d8dfedacd3b54f169972526b6a805b0d/users/20680a90b1c94164891d986feedc2b44/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:42.894829 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa9a332-1a06-4963-829a-61f293df8a42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8dfedacd3b54f169972526b6a805b0d, user_id=20680a90b1c94164891d986feedc2b44, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:42.898097 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa9a332-1a06-4963-829a-61f293df8a42 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:42.908686 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 854/1746] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:08:42 2022] PUT /identity/v3/projects/d8dfedacd3b54f169972526b6a805b0d/users/20680a90b1c94164891d986feedc2b44/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:08:42.920187 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173fa97b-6d3e-4217-9a83-009e4c807ae1 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:42.920474 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173fa97b-6d3e-4217-9a83-009e4c807ae1 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:42.920752 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173fa97b-6d3e-4217-9a83-009e4c807ae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:42.954198 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-173fa97b-6d3e-4217-9a83-009e4c807ae1 None None] MFA Rules not processed for user `20680a90b1c94164891d986feedc2b44`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:08:42.954417 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:42.954620 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-173fa97b-6d3e-4217-9a83-009e4c807ae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:43.019375 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa9a332-1a06-4963-829a-61f293df8a42 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:43.026647 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-bfa9a332-1a06-4963-829a-61f293df8a42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:43.063673 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 893/1747] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:08:42 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:08:43.063673 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967276d2-7be9-4ea7-8e63-46e48d869346 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:43.063673 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967276d2-7be9-4ea7-8e63-46e48d869346 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:43.063673 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967276d2-7be9-4ea7-8e63-46e48d869346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:43.097487 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-967276d2-7be9-4ea7-8e63-46e48d869346 None None] MFA Rules not processed for user `ced832cf86304814b020de4ed3e0955f`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:08:43.099762 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-967276d2-7be9-4ea7-8e63-46e48d869346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:43.126561 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa9a332-1a06-4963-829a-61f293df8a42 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:43.132237 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1507ff82-e4a3-49d2-9aad-ef82170cbf6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:43.132979 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1507ff82-e4a3-49d2-9aad-ef82170cbf6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:43.133334 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1507ff82-e4a3-49d2-9aad-ef82170cbf6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:43.133683 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1507ff82-e4a3-49d2-9aad-ef82170cbf6c admin admin] PATH_INFO: `/v3/projects/d3229255075a428b83187644d2f1602e` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:43.134983 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1507ff82-e4a3-49d2-9aad-ef82170cbf6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3229255075a428b83187644d2f1602e)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:43.137058 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1507ff82-e4a3-49d2-9aad-ef82170cbf6c admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:43.161074 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 894/1748] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:08:43 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:08:43.161838 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967276d2-7be9-4ea7-8e63-46e48d869346 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:43.163858 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-967276d2-7be9-4ea7-8e63-46e48d869346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:43.225673 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967276d2-7be9-4ea7-8e63-46e48d869346 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:43.244499 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-1507ff82-e4a3-49d2-9aad-ef82170cbf6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3229255075a428b83187644d2f1602e', 'request_id': 'req-1507ff82-e4a3-49d2-9aad-ef82170cbf6c'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:43.254307 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-1507ff82-e4a3-49d2-9aad-ef82170cbf6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3229255075a428b83187644d2f1602e', 'request_id': 'req-1507ff82-e4a3-49d2-9aad-ef82170cbf6c'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:43.254845 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:43.279977 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a194d1c-4cdb-442e-9b5d-bf95354d1d5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab599d98b0624a2e89090312bcab713f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:43.280601 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a194d1c-4cdb-442e-9b5d-bf95354d1d5a service cinder] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:43.280885 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a194d1c-4cdb-442e-9b5d-bf95354d1d5a service cinder] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:43.281162 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a194d1c-4cdb-442e-9b5d-bf95354d1d5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:43.336073 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-1507ff82-e4a3-49d2-9aad-ef82170cbf6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3229255075a428b83187644d2f1602e', 'request_id': 'req-1507ff82-e4a3-49d2-9aad-ef82170cbf6c'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:43.345997 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a194d1c-4cdb-442e-9b5d-bf95354d1d5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:43.345997 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a194d1c-4cdb-442e-9b5d-bf95354d1d5a service cinder] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:43.351187 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-1507ff82-e4a3-49d2-9aad-ef82170cbf6c admin admin] The token cache is being invalidate because project d3229255075a428b83187644d2f1602e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:08:43.351567 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-1507ff82-e4a3-49d2-9aad-ef82170cbf6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:43.352391 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-1507ff82-e4a3-49d2-9aad-ef82170cbf6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:43.354223 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 855/1749] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:08:43 2022] DELETE /identity/v3/projects/d3229255075a428b83187644d2f1602e => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:08:43.354338 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 895/1750] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:08:43 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:08:43.369503 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1507ff82-e4a3-49d2-9aad-ef82170cbf6c admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:43.370921 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-1507ff82-e4a3-49d2-9aad-ef82170cbf6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:43.411223 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:43.442272 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1507ff82-e4a3-49d2-9aad-ef82170cbf6c admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:43.447683 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41784d6f-a899-49d6-9559-e095d4d818ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:43.448418 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41784d6f-a899-49d6-9559-e095d4d818ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:43.449486 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41784d6f-a899-49d6-9559-e095d4d818ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:43.449795 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41784d6f-a899-49d6-9559-e095d4d818ce admin admin] PATH_INFO: `/v3/users/cf807f947fb0487eb8b45e44a206d14f` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:43.451472 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41784d6f-a899-49d6-9559-e095d4d818ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf807f947fb0487eb8b45e44a206d14f)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:43.454017 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41784d6f-a899-49d6-9559-e095d4d818ce admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:43.607592 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-41784d6f-a899-49d6-9559-e095d4d818ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf807f947fb0487eb8b45e44a206d14f'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:43.614940 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-41784d6f-a899-49d6-9559-e095d4d818ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf807f947fb0487eb8b45e44a206d14f'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:43.627568 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-41784d6f-a899-49d6-9559-e095d4d818ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf807f947fb0487eb8b45e44a206d14f'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:43.629471 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-41784d6f-a899-49d6-9559-e095d4d818ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf807f947fb0487eb8b45e44a206d14f'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:43.729329 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-41784d6f-a899-49d6-9559-e095d4d818ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf807f947fb0487eb8b45e44a206d14f'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:43.735875 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 856/1751] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:08:43 2022] DELETE /identity/v3/users/cf807f947fb0487eb8b45e44a206d14f => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:08:43.976768 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:44.086770 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:44.231986 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a194d1c-4cdb-442e-9b5d-bf95354d1d5a service cinder] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:44.246783 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a194d1c-4cdb-442e-9b5d-bf95354d1d5a service cinder] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:44.329267 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8822769e-d147-4177-b138-103e0da8de41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:44.330031 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8822769e-d147-4177-b138-103e0da8de41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:44.330375 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8822769e-d147-4177-b138-103e0da8de41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:44.330717 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8822769e-d147-4177-b138-103e0da8de41 admin admin] PATH_INFO: `/v3/projects/2a1efcc350544e2da5c8fd73ac91e55b` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:44.332037 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8822769e-d147-4177-b138-103e0da8de41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a1efcc350544e2da5c8fd73ac91e55b)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:44.334220 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8822769e-d147-4177-b138-103e0da8de41 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:44.344075 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:44.468279 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-8822769e-d147-4177-b138-103e0da8de41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a1efcc350544e2da5c8fd73ac91e55b', 'request_id': 'req-8822769e-d147-4177-b138-103e0da8de41'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:44.474433 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-8822769e-d147-4177-b138-103e0da8de41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a1efcc350544e2da5c8fd73ac91e55b', 'request_id': 'req-8822769e-d147-4177-b138-103e0da8de41'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:44.575343 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-8822769e-d147-4177-b138-103e0da8de41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a1efcc350544e2da5c8fd73ac91e55b', 'request_id': 'req-8822769e-d147-4177-b138-103e0da8de41'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:44.587117 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-8822769e-d147-4177-b138-103e0da8de41 admin admin] The token cache is being invalidate because project 2a1efcc350544e2da5c8fd73ac91e55b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81517) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:08:44.587407 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-8822769e-d147-4177-b138-103e0da8de41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:44.588066 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-8822769e-d147-4177-b138-103e0da8de41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:44.589565 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 896/1752] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:08:44 2022] DELETE /identity/v3/projects/2a1efcc350544e2da5c8fd73ac91e55b => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:08:44.594403 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9dd63b-4668-43e2-a77a-742db4c5598f None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:44.594897 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9dd63b-4668-43e2-a77a-742db4c5598f None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:44.594987 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9dd63b-4668-43e2-a77a-742db4c5598f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:44.615350 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-bb9dd63b-4668-43e2-a77a-742db4c5598f None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:08:44.616584 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-bb9dd63b-4668-43e2-a77a-742db4c5598f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:44.675480 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 857/1753] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:08:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:08:44.683439 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e72237-9935-4fa0-814a-17d67f9d8d7a None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:44.683723 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e72237-9935-4fa0-814a-17d67f9d8d7a None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:44.683992 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e72237-9935-4fa0-814a-17d67f9d8d7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:44.705132 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-18e72237-9935-4fa0-814a-17d67f9d8d7a None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:08:44.706469 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-18e72237-9935-4fa0-814a-17d67f9d8d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:44.714950 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 897/1754] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:08:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:08:44.725562 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9dd63b-4668-43e2-a77a-742db4c5598f None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:44.729026 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9dd63b-4668-43e2-a77a-742db4c5598f None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:44.733655 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a71583-f2c3-4a16-98fb-4ea37a7c6019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:44.734602 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a71583-f2c3-4a16-98fb-4ea37a7c6019 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:44.734880 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a71583-f2c3-4a16-98fb-4ea37a7c6019 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:44.735150 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a71583-f2c3-4a16-98fb-4ea37a7c6019 admin admin] PATH_INFO: `/v3/domains` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:44.735787 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a71583-f2c3-4a16-98fb-4ea37a7c6019 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 03:08:44.736103 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a71583-f2c3-4a16-98fb-4ea37a7c6019 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:44.737671 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a71583-f2c3-4a16-98fb-4ea37a7c6019 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:44.749833 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 858/1755] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Oct 14 03:08:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 03:08:44.754170 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e72237-9935-4fa0-814a-17d67f9d8d7a None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:44.764086 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e72237-9935-4fa0-814a-17d67f9d8d7a None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:44.768764 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aff00b-d0fb-4bde-8f48-ec189c13e09d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:44.769417 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aff00b-d0fb-4bde-8f48-ec189c13e09d admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:44.769701 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aff00b-d0fb-4bde-8f48-ec189c13e09d admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:44.770148 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aff00b-d0fb-4bde-8f48-ec189c13e09d admin admin] PATH_INFO: `/v3/projects` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:44.770790 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2aff00b-d0fb-4bde-8f48-ec189c13e09d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:44.772748 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2aff00b-d0fb-4bde-8f48-ec189c13e09d admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:44.891329 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 898/1756] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:08:44 2022] POST /identity/v3/projects => generated 365 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:08:44.895387 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a71583-f2c3-4a16-98fb-4ea37a7c6019 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:44.905489 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a71583-f2c3-4a16-98fb-4ea37a7c6019 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:44.964226 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8b778a-5e61-4660-828d-d62750d4fb41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:44.964838 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8b778a-5e61-4660-828d-d62750d4fb41 admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:44.965122 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8b778a-5e61-4660-828d-d62750d4fb41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:44.965396 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8b778a-5e61-4660-828d-d62750d4fb41 admin admin] PATH_INFO: `/v3/users` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:44.966094 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8b778a-5e61-4660-828d-d62750d4fb41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:44.968040 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8b778a-5e61-4660-828d-d62750d4fb41 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:45.084579 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 859/1757] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:08:44 2022] POST /identity/v3/users => generated 283 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:08:45.088605 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aff00b-d0fb-4bde-8f48-ec189c13e09d admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:45.099349 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aff00b-d0fb-4bde-8f48-ec189c13e09d admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:45.104582 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646e1b75-6309-4e0d-9e85-e716720f4b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:45.106119 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646e1b75-6309-4e0d-9e85-e716720f4b49 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:45.106119 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646e1b75-6309-4e0d-9e85-e716720f4b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:45.106119 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646e1b75-6309-4e0d-9e85-e716720f4b49 admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:45.106410 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646e1b75-6309-4e0d-9e85-e716720f4b49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:45.108015 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646e1b75-6309-4e0d-9e85-e716720f4b49 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:45.118744 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 899/1758] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:08:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:08:45.122788 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8b778a-5e61-4660-828d-d62750d4fb41 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:45.133001 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8b778a-5e61-4660-828d-d62750d4fb41 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:45.138371 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307acb1e-77c5-44f8-829e-addcb209a402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:45.138975 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307acb1e-77c5-44f8-829e-addcb209a402 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:45.139262 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307acb1e-77c5-44f8-829e-addcb209a402 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:45.139545 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307acb1e-77c5-44f8-829e-addcb209a402 admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:45.140199 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307acb1e-77c5-44f8-829e-addcb209a402 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:45.141842 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307acb1e-77c5-44f8-829e-addcb209a402 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:45.153994 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 860/1759] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:08:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:08:45.157993 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646e1b75-6309-4e0d-9e85-e716720f4b49 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:45.169471 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646e1b75-6309-4e0d-9e85-e716720f4b49 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:45.174687 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7db76db-affd-4937-b7d4-b7e39d981e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:45.175344 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7db76db-affd-4937-b7d4-b7e39d981e04 admin admin] REQUEST_METHOD: `PUT` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:45.175645 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7db76db-affd-4937-b7d4-b7e39d981e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:45.175901 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7db76db-affd-4937-b7d4-b7e39d981e04 admin admin] PATH_INFO: `/v3/projects/5c373b3fcd6e4617a914cbe866d01de2/users/a8a9136a1e2a4e19b846aa94b1d844b7/roles/9bd0470b544240209a420f640c79c06c` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:45.191136 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7db76db-affd-4937-b7d4-b7e39d981e04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c373b3fcd6e4617a914cbe866d01de2, user_id=a8a9136a1e2a4e19b846aa94b1d844b7, role_id=9bd0470b544240209a420f640c79c06c)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:45.193848 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7db76db-affd-4937-b7d4-b7e39d981e04 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:45.197179 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:45.284557 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 900/1760] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:08:45 2022] PUT /identity/v3/projects/5c373b3fcd6e4617a914cbe866d01de2/users/a8a9136a1e2a4e19b846aa94b1d844b7/roles/9bd0470b544240209a420f640c79c06c => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:08:45.295349 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7ee905-bb4b-4907-b658-c435de277551 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:45.295639 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7ee905-bb4b-4907-b658-c435de277551 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:45.295916 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7ee905-bb4b-4907-b658-c435de277551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:45.323759 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-1e7ee905-bb4b-4907-b658-c435de277551 None None] MFA Rules not processed for user `a8a9136a1e2a4e19b846aa94b1d844b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:08:45.324966 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-1e7ee905-bb4b-4907-b658-c435de277551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:45.333068 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:45.391177 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 861/1761] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:08:45 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 03:08:45.395574 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7db76db-affd-4937-b7d4-b7e39d981e04 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:45.467187 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7db76db-affd-4937-b7d4-b7e39d981e04 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:45.472164 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfd48cd-7ab8-47b3-a10b-d84d94aafe13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:45.475105 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfd48cd-7ab8-47b3-a10b-d84d94aafe13 admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:45.475105 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfd48cd-7ab8-47b3-a10b-d84d94aafe13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:45.475105 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfd48cd-7ab8-47b3-a10b-d84d94aafe13 admin admin] PATH_INFO: `/v3/projects` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:45.475105 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfd48cd-7ab8-47b3-a10b-d84d94aafe13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:45.476495 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfd48cd-7ab8-47b3-a10b-d84d94aafe13 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:45.547307 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 901/1762] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:08:45 2022] POST /identity/v3/projects => generated 365 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:08:45.548997 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7ee905-bb4b-4907-b658-c435de277551 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:45.560732 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7ee905-bb4b-4907-b658-c435de277551 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:45.621450 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6383348a-c544-4de4-8c95-84bced7e5db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:45.622200 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6383348a-c544-4de4-8c95-84bced7e5db2 admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:45.622389 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6383348a-c544-4de4-8c95-84bced7e5db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:45.622829 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6383348a-c544-4de4-8c95-84bced7e5db2 admin admin] PATH_INFO: `/v3/users` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:45.623394 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6383348a-c544-4de4-8c95-84bced7e5db2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:45.625318 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6383348a-c544-4de4-8c95-84bced7e5db2 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:45.695185 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 862/1763] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:08:45 2022] POST /identity/v3/users => generated 289 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:08:45.699218 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfd48cd-7ab8-47b3-a10b-d84d94aafe13 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:45.709490 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfd48cd-7ab8-47b3-a10b-d84d94aafe13 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:45.714831 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2959bc0-3dc0-4a8e-85dc-006a4a2472ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:45.715546 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2959bc0-3dc0-4a8e-85dc-006a4a2472ee admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:45.715831 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2959bc0-3dc0-4a8e-85dc-006a4a2472ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:45.716108 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2959bc0-3dc0-4a8e-85dc-006a4a2472ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:45.716751 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2959bc0-3dc0-4a8e-85dc-006a4a2472ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:45.718442 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2959bc0-3dc0-4a8e-85dc-006a4a2472ee admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:45.729881 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 902/1764] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:08:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:08:45.734266 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6383348a-c544-4de4-8c95-84bced7e5db2 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:45.744050 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6383348a-c544-4de4-8c95-84bced7e5db2 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:45.748653 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718cdfed-61e9-497e-9411-5d58640aeacf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:45.749297 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718cdfed-61e9-497e-9411-5d58640aeacf admin admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:45.749579 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718cdfed-61e9-497e-9411-5d58640aeacf admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:45.749895 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718cdfed-61e9-497e-9411-5d58640aeacf admin admin] PATH_INFO: `/v3/projects/5ab02ac63d06413fbceaef8d1f73b43d/users/2ebd0cc350a64174a722e6394c0cff1e/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:45.764773 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718cdfed-61e9-497e-9411-5d58640aeacf admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ab02ac63d06413fbceaef8d1f73b43d, user_id=2ebd0cc350a64174a722e6394c0cff1e, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:45.767539 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718cdfed-61e9-497e-9411-5d58640aeacf admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:45.843874 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 863/1765] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:08:45 2022] PUT /identity/v3/projects/5ab02ac63d06413fbceaef8d1f73b43d/users/2ebd0cc350a64174a722e6394c0cff1e/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:08:45.847804 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2959bc0-3dc0-4a8e-85dc-006a4a2472ee admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:45.858162 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2959bc0-3dc0-4a8e-85dc-006a4a2472ee admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:45.914827 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44daaa83-1fac-459b-9475-3db943cefa36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:45.915424 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44daaa83-1fac-459b-9475-3db943cefa36 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:45.915707 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44daaa83-1fac-459b-9475-3db943cefa36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:45.915980 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44daaa83-1fac-459b-9475-3db943cefa36 admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:45.916647 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44daaa83-1fac-459b-9475-3db943cefa36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:45.918313 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44daaa83-1fac-459b-9475-3db943cefa36 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:45.927756 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 903/1766] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:08:45 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 03:08:45.931957 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718cdfed-61e9-497e-9411-5d58640aeacf admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:45.942413 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718cdfed-61e9-497e-9411-5d58640aeacf admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:45.950109 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4ac23b-3496-4643-952e-b43f67160f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:45.950109 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4ac23b-3496-4643-952e-b43f67160f8d admin admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:45.950109 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4ac23b-3496-4643-952e-b43f67160f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:45.950109 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4ac23b-3496-4643-952e-b43f67160f8d admin admin] PATH_INFO: `/v3/projects/5ab02ac63d06413fbceaef8d1f73b43d/users/2ebd0cc350a64174a722e6394c0cff1e/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:45.950850 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4ac23b-3496-4643-952e-b43f67160f8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ab02ac63d06413fbceaef8d1f73b43d, user_id=2ebd0cc350a64174a722e6394c0cff1e, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:45.953630 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4ac23b-3496-4643-952e-b43f67160f8d admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:45.964449 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 864/1767] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:08:45 2022] PUT /identity/v3/projects/5ab02ac63d06413fbceaef8d1f73b43d/users/2ebd0cc350a64174a722e6394c0cff1e/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:08:45.975418 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69759f0-f60c-43fc-8e3c-3ebae89b3d47 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:45.975706 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69759f0-f60c-43fc-8e3c-3ebae89b3d47 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:45.975983 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69759f0-f60c-43fc-8e3c-3ebae89b3d47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:46.003677 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-b69759f0-f60c-43fc-8e3c-3ebae89b3d47 None None] MFA Rules not processed for user `2ebd0cc350a64174a722e6394c0cff1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:08:46.004889 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-b69759f0-f60c-43fc-8e3c-3ebae89b3d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:46.058987 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:46.077302 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 904/1768] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:08:45 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:08:46.089699 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4ac23b-3496-4643-952e-b43f67160f8d admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:46.091426 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-8a4ac23b-3496-4643-952e-b43f67160f8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:46.154737 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4ac23b-3496-4643-952e-b43f67160f8d admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:46.158324 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2924b185-a8a8-4cbf-b41f-27d81ed9f4ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab599d98b0624a2e89090312bcab713f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:46.158927 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2924b185-a8a8-4cbf-b41f-27d81ed9f4ca service cinder] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:46.159206 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2924b185-a8a8-4cbf-b41f-27d81ed9f4ca service cinder] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:46.159478 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2924b185-a8a8-4cbf-b41f-27d81ed9f4ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:46.168410 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2924b185-a8a8-4cbf-b41f-27d81ed9f4ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:46.169860 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2924b185-a8a8-4cbf-b41f-27d81ed9f4ca service cinder] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:46.179186 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 865/1769] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:08:46 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:08:46.453312 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69759f0-f60c-43fc-8e3c-3ebae89b3d47 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:46.453969 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2924b185-a8a8-4cbf-b41f-27d81ed9f4ca service cinder] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:46.454885 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-b69759f0-f60c-43fc-8e3c-3ebae89b3d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:46.455424 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-2924b185-a8a8-4cbf-b41f-27d81ed9f4ca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:46.463465 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:46.466983 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2924b185-a8a8-4cbf-b41f-27d81ed9f4ca service cinder] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:46.472382 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a78a4c-9245-40df-b646-5e09fb019791 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab599d98b0624a2e89090312bcab713f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:46.472382 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a78a4c-9245-40df-b646-5e09fb019791 service cinder] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:46.472382 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a78a4c-9245-40df-b646-5e09fb019791 service cinder] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:46.472382 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a78a4c-9245-40df-b646-5e09fb019791 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:46.527712 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69759f0-f60c-43fc-8e3c-3ebae89b3d47 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:46.532663 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b13c3a7-cf77-408a-9b15-d1dc54cd9dfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:46.533303 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b13c3a7-cf77-408a-9b15-d1dc54cd9dfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:46.533709 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b13c3a7-cf77-408a-9b15-d1dc54cd9dfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:46.533895 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b13c3a7-cf77-408a-9b15-d1dc54cd9dfa admin admin] PATH_INFO: `/v3/users/8e1de3646bfa4231ad5d3ed57a9fd028` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:46.535043 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b13c3a7-cf77-408a-9b15-d1dc54cd9dfa admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e1de3646bfa4231ad5d3ed57a9fd028)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:46.537052 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b13c3a7-cf77-408a-9b15-d1dc54cd9dfa admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:46.537590 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a78a4c-9245-40df-b646-5e09fb019791 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:46.539233 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a78a4c-9245-40df-b646-5e09fb019791 service cinder] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:46.549526 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 866/1770] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:08:46 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 03:08:46.754003 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-9b13c3a7-cf77-408a-9b15-d1dc54cd9dfa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e1de3646bfa4231ad5d3ed57a9fd028'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:46.754732 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-9b13c3a7-cf77-408a-9b15-d1dc54cd9dfa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e1de3646bfa4231ad5d3ed57a9fd028'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:46.858420 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-9b13c3a7-cf77-408a-9b15-d1dc54cd9dfa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e1de3646bfa4231ad5d3ed57a9fd028'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:46.859430 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-9b13c3a7-cf77-408a-9b15-d1dc54cd9dfa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e1de3646bfa4231ad5d3ed57a9fd028'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:46.872161 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-9b13c3a7-cf77-408a-9b15-d1dc54cd9dfa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e1de3646bfa4231ad5d3ed57a9fd028'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:46.900009 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 905/1771] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:08:46 2022] DELETE /identity/v3/users/8e1de3646bfa4231ad5d3ed57a9fd028 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:08:46.904301 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab53eed-c048-4c4f-aac0-d2f351bdb0fa None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:46.904782 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab53eed-c048-4c4f-aac0-d2f351bdb0fa None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:46.905022 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab53eed-c048-4c4f-aac0-d2f351bdb0fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:46.926120 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-6ab53eed-c048-4c4f-aac0-d2f351bdb0fa None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:08:46.927336 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-6ab53eed-c048-4c4f-aac0-d2f351bdb0fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:47.007462 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 867/1772] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:08:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 03:08:47.023871 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b13c3a7-cf77-408a-9b15-d1dc54cd9dfa admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:47.025543 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-9b13c3a7-cf77-408a-9b15-d1dc54cd9dfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:47.097445 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b13c3a7-cf77-408a-9b15-d1dc54cd9dfa admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:47.101974 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8732b1a0-a429-48f4-8c90-756f569b7ab6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:47.104091 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8732b1a0-a429-48f4-8c90-756f569b7ab6 service neutron] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:47.104091 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8732b1a0-a429-48f4-8c90-756f569b7ab6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:47.104091 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8732b1a0-a429-48f4-8c90-756f569b7ab6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:47.120742 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8732b1a0-a429-48f4-8c90-756f569b7ab6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:47.123443 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8732b1a0-a429-48f4-8c90-756f569b7ab6 service neutron] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:47.136044 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 906/1773] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:08:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:08:47.598061 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab53eed-c048-4c4f-aac0-d2f351bdb0fa None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:47.599714 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-6ab53eed-c048-4c4f-aac0-d2f351bdb0fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:47.618461 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab53eed-c048-4c4f-aac0-d2f351bdb0fa None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:47.622171 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7c3978-f5f4-462f-aba0-f3ece69ad2c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:47.622907 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c3978-f5f4-462f-aba0-f3ece69ad2c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:47.623324 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c3978-f5f4-462f-aba0-f3ece69ad2c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:47.623673 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c3978-f5f4-462f-aba0-f3ece69ad2c6 admin admin] PATH_INFO: `/v3/projects/b7e8f1717dd94efa843abbb993735c84` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:47.625060 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7c3978-f5f4-462f-aba0-f3ece69ad2c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7e8f1717dd94efa843abbb993735c84)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:47.633188 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7c3978-f5f4-462f-aba0-f3ece69ad2c6 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:47.767437 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-ba7c3978-f5f4-462f-aba0-f3ece69ad2c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7e8f1717dd94efa843abbb993735c84', 'request_id': 'req-ba7c3978-f5f4-462f-aba0-f3ece69ad2c6'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:47.773942 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-ba7c3978-f5f4-462f-aba0-f3ece69ad2c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7e8f1717dd94efa843abbb993735c84', 'request_id': 'req-ba7c3978-f5f4-462f-aba0-f3ece69ad2c6'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:47.862061 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-ba7c3978-f5f4-462f-aba0-f3ece69ad2c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7e8f1717dd94efa843abbb993735c84', 'request_id': 'req-ba7c3978-f5f4-462f-aba0-f3ece69ad2c6'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:47.875560 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-ba7c3978-f5f4-462f-aba0-f3ece69ad2c6 admin admin] The token cache is being invalidate because project b7e8f1717dd94efa843abbb993735c84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:08:47.875844 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-ba7c3978-f5f4-462f-aba0-f3ece69ad2c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:47.876559 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-ba7c3978-f5f4-462f-aba0-f3ece69ad2c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:47.878341 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 868/1774] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:08:47 2022] DELETE /identity/v3/projects/b7e8f1717dd94efa843abbb993735c84 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:08:47.882999 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8732b1a0-a429-48f4-8c90-756f569b7ab6 service neutron] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:47.893434 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8732b1a0-a429-48f4-8c90-756f569b7ab6 service neutron] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:47.907244 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:47.974612 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32aa9d3-d98e-408c-bfaf-3d97420dabdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:47.975920 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32aa9d3-d98e-408c-bfaf-3d97420dabdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:47.975920 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32aa9d3-d98e-408c-bfaf-3d97420dabdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:47.976147 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32aa9d3-d98e-408c-bfaf-3d97420dabdf admin admin] PATH_INFO: `/v3/users/20680a90b1c94164891d986feedc2b44` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:47.977363 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32aa9d3-d98e-408c-bfaf-3d97420dabdf admin admin] RBAC: Authorizing `identity:delete_user(user_id=20680a90b1c94164891d986feedc2b44)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:47.980291 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32aa9d3-d98e-408c-bfaf-3d97420dabdf admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:48.181656 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-e32aa9d3-d98e-408c-bfaf-3d97420dabdf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20680a90b1c94164891d986feedc2b44'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:48.182283 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-e32aa9d3-d98e-408c-bfaf-3d97420dabdf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20680a90b1c94164891d986feedc2b44'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:48.261489 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-e32aa9d3-d98e-408c-bfaf-3d97420dabdf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20680a90b1c94164891d986feedc2b44'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:48.262285 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-e32aa9d3-d98e-408c-bfaf-3d97420dabdf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20680a90b1c94164891d986feedc2b44'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:48.289140 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-e32aa9d3-d98e-408c-bfaf-3d97420dabdf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20680a90b1c94164891d986feedc2b44'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:48.334076 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 907/1775] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:08:47 2022] DELETE /identity/v3/users/20680a90b1c94164891d986feedc2b44 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 03:08:48.989883 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7c3978-f5f4-462f-aba0-f3ece69ad2c6 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:48.991663 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-ba7c3978-f5f4-462f-aba0-f3ece69ad2c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:49.013034 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7c3978-f5f4-462f-aba0-f3ece69ad2c6 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:49.040033 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:49.081290 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c5ed31-4b6f-4369-b368-2fa37e0e993c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:49.081290 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c5ed31-4b6f-4369-b368-2fa37e0e993c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:49.081290 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c5ed31-4b6f-4369-b368-2fa37e0e993c admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:49.081290 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c5ed31-4b6f-4369-b368-2fa37e0e993c admin admin] PATH_INFO: `/v3/projects/d8dfedacd3b54f169972526b6a805b0d` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:49.081607 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c5ed31-4b6f-4369-b368-2fa37e0e993c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8dfedacd3b54f169972526b6a805b0d)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:49.081607 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c5ed31-4b6f-4369-b368-2fa37e0e993c admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:49.235558 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-62c5ed31-4b6f-4369-b368-2fa37e0e993c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8dfedacd3b54f169972526b6a805b0d', 'request_id': 'req-62c5ed31-4b6f-4369-b368-2fa37e0e993c'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:49.242759 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-62c5ed31-4b6f-4369-b368-2fa37e0e993c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8dfedacd3b54f169972526b6a805b0d', 'request_id': 'req-62c5ed31-4b6f-4369-b368-2fa37e0e993c'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:49.324338 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-62c5ed31-4b6f-4369-b368-2fa37e0e993c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8dfedacd3b54f169972526b6a805b0d', 'request_id': 'req-62c5ed31-4b6f-4369-b368-2fa37e0e993c'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:49.339862 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-62c5ed31-4b6f-4369-b368-2fa37e0e993c admin admin] The token cache is being invalidate because project d8dfedacd3b54f169972526b6a805b0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:08:49.340319 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-62c5ed31-4b6f-4369-b368-2fa37e0e993c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:49.341040 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-62c5ed31-4b6f-4369-b368-2fa37e0e993c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:49.342846 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 869/1776] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:08:48 2022] DELETE /identity/v3/projects/d8dfedacd3b54f169972526b6a805b0d => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:08:49.347821 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ee9a1a-84e9-4f79-9017-8098962f4901 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:49.348449 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ee9a1a-84e9-4f79-9017-8098962f4901 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:49.348449 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ee9a1a-84e9-4f79-9017-8098962f4901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:49.372233 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-e7ee9a1a-84e9-4f79-9017-8098962f4901 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:08:49.373965 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-e7ee9a1a-84e9-4f79-9017-8098962f4901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:49.433975 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 908/1777] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:08:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:08:49.441986 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e426a269-bf42-4983-8c88-f6240cc97b86 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:49.442505 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e426a269-bf42-4983-8c88-f6240cc97b86 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:49.442792 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e426a269-bf42-4983-8c88-f6240cc97b86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:49.468004 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-e426a269-bf42-4983-8c88-f6240cc97b86 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:08:49.469383 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-e426a269-bf42-4983-8c88-f6240cc97b86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:49.478054 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 870/1778] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:08:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:08:49.482259 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ee9a1a-84e9-4f79-9017-8098962f4901 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:49.501746 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ee9a1a-84e9-4f79-9017-8098962f4901 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:49.501746 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6595e259-d8b8-450e-8792-b1c616b12d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:49.501746 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6595e259-d8b8-450e-8792-b1c616b12d09 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:49.501746 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6595e259-d8b8-450e-8792-b1c616b12d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:49.502419 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6595e259-d8b8-450e-8792-b1c616b12d09 admin admin] PATH_INFO: `/v3/domains` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:49.502419 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6595e259-d8b8-450e-8792-b1c616b12d09 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 03:08:49.502419 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6595e259-d8b8-450e-8792-b1c616b12d09 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:49.503400 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6595e259-d8b8-450e-8792-b1c616b12d09 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:49.523808 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 909/1779] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Oct 14 03:08:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 03:08:49.525922 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e426a269-bf42-4983-8c88-f6240cc97b86 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:49.537854 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e426a269-bf42-4983-8c88-f6240cc97b86 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:49.543406 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0d5a63-f5d8-4791-b89b-4476e386a8b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:49.544285 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0d5a63-f5d8-4791-b89b-4476e386a8b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:49.544571 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0d5a63-f5d8-4791-b89b-4476e386a8b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:49.544847 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0d5a63-f5d8-4791-b89b-4476e386a8b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:49.545797 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0d5a63-f5d8-4791-b89b-4476e386a8b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:49.548007 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0d5a63-f5d8-4791-b89b-4476e386a8b0 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:49.683931 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 871/1780] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:08:49 2022] POST /identity/v3/projects => generated 375 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:08:49.688260 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6595e259-d8b8-450e-8792-b1c616b12d09 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:49.699020 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6595e259-d8b8-450e-8792-b1c616b12d09 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:49.721605 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:49.781798 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f832ffeb-410e-464a-883e-47f69136afb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:49.785842 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f832ffeb-410e-464a-883e-47f69136afb4 admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:49.785842 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f832ffeb-410e-464a-883e-47f69136afb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:49.785842 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f832ffeb-410e-464a-883e-47f69136afb4 admin admin] PATH_INFO: `/v3/users` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:49.785842 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f832ffeb-410e-464a-883e-47f69136afb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:49.785842 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f832ffeb-410e-464a-883e-47f69136afb4 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:49.846346 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 910/1781] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:08:49 2022] POST /identity/v3/users => generated 288 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:08:49.851159 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0d5a63-f5d8-4791-b89b-4476e386a8b0 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:49.864005 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0d5a63-f5d8-4791-b89b-4476e386a8b0 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:49.871348 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dc9bf8-518e-45cf-8103-e40536ca1211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:49.872024 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc9bf8-518e-45cf-8103-e40536ca1211 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:49.872310 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc9bf8-518e-45cf-8103-e40536ca1211 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:49.873050 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc9bf8-518e-45cf-8103-e40536ca1211 admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:49.873428 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dc9bf8-518e-45cf-8103-e40536ca1211 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:49.875093 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dc9bf8-518e-45cf-8103-e40536ca1211 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:49.889568 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 872/1782] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:08:49 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 03:08:49.890664 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f832ffeb-410e-464a-883e-47f69136afb4 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:49.900767 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f832ffeb-410e-464a-883e-47f69136afb4 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:49.905563 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa33f27-a748-403e-afa9-47d1dfbd8bde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:49.906284 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa33f27-a748-403e-afa9-47d1dfbd8bde admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:49.906627 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa33f27-a748-403e-afa9-47d1dfbd8bde admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:49.906964 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa33f27-a748-403e-afa9-47d1dfbd8bde admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:49.907674 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa33f27-a748-403e-afa9-47d1dfbd8bde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:49.909340 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa33f27-a748-403e-afa9-47d1dfbd8bde admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:49.920956 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 911/1783] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:08:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 03:08:49.925266 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dc9bf8-518e-45cf-8103-e40536ca1211 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:49.936045 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dc9bf8-518e-45cf-8103-e40536ca1211 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:49.941488 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c305e6-ce4f-4c37-835e-917fff3afe8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:49.941488 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c305e6-ce4f-4c37-835e-917fff3afe8b admin admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:49.941690 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c305e6-ce4f-4c37-835e-917fff3afe8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:49.941933 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c305e6-ce4f-4c37-835e-917fff3afe8b admin admin] PATH_INFO: `/v3/projects/415712dea1e842ffa7eaf3035cac0fb9/users/82f2b4ee47fc45aca5cd328178bff4fd/roles/9bd0470b544240209a420f640c79c06c` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:49.958064 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c305e6-ce4f-4c37-835e-917fff3afe8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=415712dea1e842ffa7eaf3035cac0fb9, user_id=82f2b4ee47fc45aca5cd328178bff4fd, role_id=9bd0470b544240209a420f640c79c06c)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:49.961357 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c305e6-ce4f-4c37-835e-917fff3afe8b admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:50.038560 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 873/1784] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:08:49 2022] PUT /identity/v3/projects/415712dea1e842ffa7eaf3035cac0fb9/users/82f2b4ee47fc45aca5cd328178bff4fd/roles/9bd0470b544240209a420f640c79c06c => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:08:50.051338 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441535bc-30a6-424d-bcf9-b6c451952aa0 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:50.051628 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441535bc-30a6-424d-bcf9-b6c451952aa0 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:50.051905 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441535bc-30a6-424d-bcf9-b6c451952aa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:50.085498 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-441535bc-30a6-424d-bcf9-b6c451952aa0 None None] MFA Rules not processed for user `82f2b4ee47fc45aca5cd328178bff4fd`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:08:50.086888 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-441535bc-30a6-424d-bcf9-b6c451952aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:50.163707 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 912/1785] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:08:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:08:50.174750 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c305e6-ce4f-4c37-835e-917fff3afe8b admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:50.176542 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-15c305e6-ce4f-4c37-835e-917fff3afe8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:50.251684 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c305e6-ce4f-4c37-835e-917fff3afe8b admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:50.258137 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d109a492-d83f-4054-ba91-18a04423115f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab599d98b0624a2e89090312bcab713f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:50.258137 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d109a492-d83f-4054-ba91-18a04423115f service cinder] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:50.258137 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d109a492-d83f-4054-ba91-18a04423115f service cinder] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:50.258137 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d109a492-d83f-4054-ba91-18a04423115f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:50.265039 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:50.269003 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d109a492-d83f-4054-ba91-18a04423115f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:50.271147 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d109a492-d83f-4054-ba91-18a04423115f service cinder] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:50.292268 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 874/1786] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:08:50 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:08:53.847820 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441535bc-30a6-424d-bcf9-b6c451952aa0 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:53.850027 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-441535bc-30a6-424d-bcf9-b6c451952aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:53.936442 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441535bc-30a6-424d-bcf9-b6c451952aa0 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:53.948385 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39cf2c9-af56-4ee4-b6d7-73104a2202e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:53.949107 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39cf2c9-af56-4ee4-b6d7-73104a2202e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:53.949458 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39cf2c9-af56-4ee4-b6d7-73104a2202e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:53.951071 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39cf2c9-af56-4ee4-b6d7-73104a2202e1 admin admin] PATH_INFO: `/v3/users/a8a9136a1e2a4e19b846aa94b1d844b7` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:53.952316 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39cf2c9-af56-4ee4-b6d7-73104a2202e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8a9136a1e2a4e19b846aa94b1d844b7)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:53.954789 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39cf2c9-af56-4ee4-b6d7-73104a2202e1 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:54.121202 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:54.149166 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-f39cf2c9-af56-4ee4-b6d7-73104a2202e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8a9136a1e2a4e19b846aa94b1d844b7'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:54.150604 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-f39cf2c9-af56-4ee4-b6d7-73104a2202e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8a9136a1e2a4e19b846aa94b1d844b7'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:54.227400 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-f39cf2c9-af56-4ee4-b6d7-73104a2202e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8a9136a1e2a4e19b846aa94b1d844b7'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:54.228105 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-f39cf2c9-af56-4ee4-b6d7-73104a2202e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8a9136a1e2a4e19b846aa94b1d844b7'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:54.236834 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-f39cf2c9-af56-4ee4-b6d7-73104a2202e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8a9136a1e2a4e19b846aa94b1d844b7'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:54.259406 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 913/1787] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:08:53 2022] DELETE /identity/v3/users/a8a9136a1e2a4e19b846aa94b1d844b7 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:08:54.263827 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7716171e-6050-43a5-80b6-6d1c6e810dd3 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:54.264379 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7716171e-6050-43a5-80b6-6d1c6e810dd3 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:54.264670 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7716171e-6050-43a5-80b6-6d1c6e810dd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:54.290237 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-7716171e-6050-43a5-80b6-6d1c6e810dd3 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:08:54.291977 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-7716171e-6050-43a5-80b6-6d1c6e810dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:54.375825 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 875/1788] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:08:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:08:54.381309 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39cf2c9-af56-4ee4-b6d7-73104a2202e1 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:54.381492 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7716171e-6050-43a5-80b6-6d1c6e810dd3 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:54.383240 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-7716171e-6050-43a5-80b6-6d1c6e810dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:54.383389 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-f39cf2c9-af56-4ee4-b6d7-73104a2202e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:54.403320 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39cf2c9-af56-4ee4-b6d7-73104a2202e1 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:54.408854 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f72d9d-969d-4e11-ad69-a1590a9256a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:54.409868 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f72d9d-969d-4e11-ad69-a1590a9256a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:54.410280 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f72d9d-969d-4e11-ad69-a1590a9256a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:54.410638 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f72d9d-969d-4e11-ad69-a1590a9256a4 admin admin] PATH_INFO: `/v3/users/82f2b4ee47fc45aca5cd328178bff4fd` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:54.412018 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f72d9d-969d-4e11-ad69-a1590a9256a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82f2b4ee47fc45aca5cd328178bff4fd)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:54.414803 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f72d9d-969d-4e11-ad69-a1590a9256a4 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:54.453353 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7716171e-6050-43a5-80b6-6d1c6e810dd3 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:54.457484 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ec6a1c-6e6c-49ae-9e74-10c3ff921379 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:54.458451 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ec6a1c-6e6c-49ae-9e74-10c3ff921379 service neutron] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:54.458682 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ec6a1c-6e6c-49ae-9e74-10c3ff921379 service neutron] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:54.458964 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ec6a1c-6e6c-49ae-9e74-10c3ff921379 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:54.470179 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ec6a1c-6e6c-49ae-9e74-10c3ff921379 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:54.471767 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ec6a1c-6e6c-49ae-9e74-10c3ff921379 service neutron] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:54.483438 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 876/1789] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:08:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 03:08:54.635104 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-f3f72d9d-969d-4e11-ad69-a1590a9256a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82f2b4ee47fc45aca5cd328178bff4fd'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:54.636043 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-f3f72d9d-969d-4e11-ad69-a1590a9256a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82f2b4ee47fc45aca5cd328178bff4fd'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:54.718070 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-f3f72d9d-969d-4e11-ad69-a1590a9256a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82f2b4ee47fc45aca5cd328178bff4fd'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:54.718725 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-f3f72d9d-969d-4e11-ad69-a1590a9256a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82f2b4ee47fc45aca5cd328178bff4fd'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:54.732605 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-f3f72d9d-969d-4e11-ad69-a1590a9256a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82f2b4ee47fc45aca5cd328178bff4fd'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:54.794671 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 914/1790] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:08:54 2022] DELETE /identity/v3/users/82f2b4ee47fc45aca5cd328178bff4fd => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:08:54.796332 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab7d469-ae37-4b37-bf6e-8c975a050062 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:54.796629 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab7d469-ae37-4b37-bf6e-8c975a050062 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:54.796909 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab7d469-ae37-4b37-bf6e-8c975a050062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:54.849311 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-1ab7d469-ae37-4b37-bf6e-8c975a050062 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:08:54.849311 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-1ab7d469-ae37-4b37-bf6e-8c975a050062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:54.973339 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 877/1791] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:08:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:08:54.981213 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f72d9d-969d-4e11-ad69-a1590a9256a4 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:54.982706 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-f3f72d9d-969d-4e11-ad69-a1590a9256a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:55.051684 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f72d9d-969d-4e11-ad69-a1590a9256a4 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:55.055730 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e5454c-7d75-4b0f-89a9-e8222af9c6ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:55.058899 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e5454c-7d75-4b0f-89a9-e8222af9c6ef service neutron] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:55.058899 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e5454c-7d75-4b0f-89a9-e8222af9c6ef service neutron] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:55.058899 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e5454c-7d75-4b0f-89a9-e8222af9c6ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:55.067089 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e5454c-7d75-4b0f-89a9-e8222af9c6ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:55.068527 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e5454c-7d75-4b0f-89a9-e8222af9c6ef service neutron] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:55.079003 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 915/1792] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:08:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:08:55.087518 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab7d469-ae37-4b37-bf6e-8c975a050062 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:55.088950 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-1ab7d469-ae37-4b37-bf6e-8c975a050062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:55.102172 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab7d469-ae37-4b37-bf6e-8c975a050062 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:55.107281 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26c260e-cd0c-405f-adb2-debdeb70ec3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:55.107935 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26c260e-cd0c-405f-adb2-debdeb70ec3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:55.108218 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26c260e-cd0c-405f-adb2-debdeb70ec3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:55.108497 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26c260e-cd0c-405f-adb2-debdeb70ec3a admin admin] PATH_INFO: `/v3/projects/5c373b3fcd6e4617a914cbe866d01de2` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:55.110353 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26c260e-cd0c-405f-adb2-debdeb70ec3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c373b3fcd6e4617a914cbe866d01de2)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:55.112529 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26c260e-cd0c-405f-adb2-debdeb70ec3a admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:55.218475 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:55.230049 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-a26c260e-cd0c-405f-adb2-debdeb70ec3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c373b3fcd6e4617a914cbe866d01de2', 'request_id': 'req-a26c260e-cd0c-405f-adb2-debdeb70ec3a'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:55.236983 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-a26c260e-cd0c-405f-adb2-debdeb70ec3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c373b3fcd6e4617a914cbe866d01de2', 'request_id': 'req-a26c260e-cd0c-405f-adb2-debdeb70ec3a'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:55.324873 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-a26c260e-cd0c-405f-adb2-debdeb70ec3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c373b3fcd6e4617a914cbe866d01de2', 'request_id': 'req-a26c260e-cd0c-405f-adb2-debdeb70ec3a'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:55.338724 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-a26c260e-cd0c-405f-adb2-debdeb70ec3a admin admin] The token cache is being invalidate because project 5c373b3fcd6e4617a914cbe866d01de2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:08:55.339340 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-a26c260e-cd0c-405f-adb2-debdeb70ec3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:55.340248 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-a26c260e-cd0c-405f-adb2-debdeb70ec3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:55.346351 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e5454c-7d75-4b0f-89a9-e8222af9c6ef service neutron] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:55.347235 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 878/1793] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:08:55 2022] DELETE /identity/v3/projects/5c373b3fcd6e4617a914cbe866d01de2 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 03:08:55.357830 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e5454c-7d75-4b0f-89a9-e8222af9c6ef service neutron] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:55.383816 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:55.427414 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96201b39-cbda-4aa9-afc2-a4a3ab5c7cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:55.428079 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96201b39-cbda-4aa9-afc2-a4a3ab5c7cf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:55.428361 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96201b39-cbda-4aa9-afc2-a4a3ab5c7cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:55.428632 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96201b39-cbda-4aa9-afc2-a4a3ab5c7cf6 admin admin] PATH_INFO: `/v3/users/2ebd0cc350a64174a722e6394c0cff1e` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:55.430709 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96201b39-cbda-4aa9-afc2-a4a3ab5c7cf6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ebd0cc350a64174a722e6394c0cff1e)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:55.432587 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96201b39-cbda-4aa9-afc2-a4a3ab5c7cf6 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:55.644483 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-96201b39-cbda-4aa9-afc2-a4a3ab5c7cf6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ebd0cc350a64174a722e6394c0cff1e'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:55.646160 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-96201b39-cbda-4aa9-afc2-a4a3ab5c7cf6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ebd0cc350a64174a722e6394c0cff1e'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:55.724107 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-96201b39-cbda-4aa9-afc2-a4a3ab5c7cf6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ebd0cc350a64174a722e6394c0cff1e'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:55.724817 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-96201b39-cbda-4aa9-afc2-a4a3ab5c7cf6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ebd0cc350a64174a722e6394c0cff1e'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:55.736393 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-96201b39-cbda-4aa9-afc2-a4a3ab5c7cf6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ebd0cc350a64174a722e6394c0cff1e'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:55.764244 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 916/1794] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:08:55 2022] DELETE /identity/v3/users/2ebd0cc350a64174a722e6394c0cff1e => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:08:55.955175 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26c260e-cd0c-405f-adb2-debdeb70ec3a admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:55.956846 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-a26c260e-cd0c-405f-adb2-debdeb70ec3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:56.008976 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26c260e-cd0c-405f-adb2-debdeb70ec3a admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:56.114994 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574e1c83-a2bc-417a-9004-910bd669edf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:56.115962 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574e1c83-a2bc-417a-9004-910bd669edf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:56.116388 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574e1c83-a2bc-417a-9004-910bd669edf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:56.117441 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574e1c83-a2bc-417a-9004-910bd669edf1 admin admin] PATH_INFO: `/v3/projects/415712dea1e842ffa7eaf3035cac0fb9` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:56.119073 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574e1c83-a2bc-417a-9004-910bd669edf1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=415712dea1e842ffa7eaf3035cac0fb9)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:56.125332 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574e1c83-a2bc-417a-9004-910bd669edf1 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:56.261451 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-574e1c83-a2bc-417a-9004-910bd669edf1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '415712dea1e842ffa7eaf3035cac0fb9', 'request_id': 'req-574e1c83-a2bc-417a-9004-910bd669edf1'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:56.270045 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-574e1c83-a2bc-417a-9004-910bd669edf1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '415712dea1e842ffa7eaf3035cac0fb9', 'request_id': 'req-574e1c83-a2bc-417a-9004-910bd669edf1'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:56.344132 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-574e1c83-a2bc-417a-9004-910bd669edf1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '415712dea1e842ffa7eaf3035cac0fb9', 'request_id': 'req-574e1c83-a2bc-417a-9004-910bd669edf1'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:56.349896 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96201b39-cbda-4aa9-afc2-a4a3ab5c7cf6 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:56.352365 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-96201b39-cbda-4aa9-afc2-a4a3ab5c7cf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:56.365322 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-574e1c83-a2bc-417a-9004-910bd669edf1 admin admin] The token cache is being invalidate because project 415712dea1e842ffa7eaf3035cac0fb9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:08:56.365872 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-574e1c83-a2bc-417a-9004-910bd669edf1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:56.367523 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-574e1c83-a2bc-417a-9004-910bd669edf1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:56.379456 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96201b39-cbda-4aa9-afc2-a4a3ab5c7cf6 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:56.390529 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 879/1795] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:08:55 2022] DELETE /identity/v3/projects/415712dea1e842ffa7eaf3035cac0fb9 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:08:56.395333 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a481c76a-cb34-4d86-b4ea-47e1128e325b None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:56.396612 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a481c76a-cb34-4d86-b4ea-47e1128e325b None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:56.397039 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a481c76a-cb34-4d86-b4ea-47e1128e325b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:56.421020 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-a481c76a-cb34-4d86-b4ea-47e1128e325b None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:08:56.423822 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-a481c76a-cb34-4d86-b4ea-47e1128e325b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:56.463730 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5f561a-5cd4-4a98-835a-996d4d0842f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:56.464438 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5f561a-5cd4-4a98-835a-996d4d0842f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:56.464810 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5f561a-5cd4-4a98-835a-996d4d0842f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:56.465254 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5f561a-5cd4-4a98-835a-996d4d0842f9 admin admin] PATH_INFO: `/v3/projects/5ab02ac63d06413fbceaef8d1f73b43d` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:56.466790 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5f561a-5cd4-4a98-835a-996d4d0842f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ab02ac63d06413fbceaef8d1f73b43d)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:56.470379 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5f561a-5cd4-4a98-835a-996d4d0842f9 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:56.494131 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:56.511872 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 880/1796] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:08:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 03:08:56.580377 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12548071-bb81-4380-adc6-1f7e9e3580fd None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:56.580655 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12548071-bb81-4380-adc6-1f7e9e3580fd None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:56.580931 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12548071-bb81-4380-adc6-1f7e9e3580fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:56.600549 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-12548071-bb81-4380-adc6-1f7e9e3580fd None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:08:56.602316 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-12548071-bb81-4380-adc6-1f7e9e3580fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:56.613806 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 881/1797] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:08:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 03:08:56.617441 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12548071-bb81-4380-adc6-1f7e9e3580fd None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:56.626137 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:56.629154 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-aa5f561a-5cd4-4a98-835a-996d4d0842f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ab02ac63d06413fbceaef8d1f73b43d', 'request_id': 'req-aa5f561a-5cd4-4a98-835a-996d4d0842f9'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:56.629629 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12548071-bb81-4380-adc6-1f7e9e3580fd None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:56.635841 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-aa5f561a-5cd4-4a98-835a-996d4d0842f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ab02ac63d06413fbceaef8d1f73b43d', 'request_id': 'req-aa5f561a-5cd4-4a98-835a-996d4d0842f9'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:56.700594 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccebd15-c984-4e57-aef8-329128bb254f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:56.701235 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccebd15-c984-4e57-aef8-329128bb254f admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:56.701522 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccebd15-c984-4e57-aef8-329128bb254f admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:56.701824 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccebd15-c984-4e57-aef8-329128bb254f admin admin] PATH_INFO: `/v3/domains` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:56.702551 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccebd15-c984-4e57-aef8-329128bb254f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 03:08:56.703084 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccebd15-c984-4e57-aef8-329128bb254f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:56.704723 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccebd15-c984-4e57-aef8-329128bb254f admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:56.712748 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-aa5f561a-5cd4-4a98-835a-996d4d0842f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ab02ac63d06413fbceaef8d1f73b43d', 'request_id': 'req-aa5f561a-5cd4-4a98-835a-996d4d0842f9'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:56.731673 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 882/1798] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Oct 14 03:08:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 03:08:56.733448 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccebd15-c984-4e57-aef8-329128bb254f admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:56.739926 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-aa5f561a-5cd4-4a98-835a-996d4d0842f9 admin admin] The token cache is being invalidate because project 5ab02ac63d06413fbceaef8d1f73b43d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81517) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:08:56.740525 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-aa5f561a-5cd4-4a98-835a-996d4d0842f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:56.741417 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-aa5f561a-5cd4-4a98-835a-996d4d0842f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:56.751431 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 917/1799] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:08:56 2022] DELETE /identity/v3/projects/5ab02ac63d06413fbceaef8d1f73b43d => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:08:56.751769 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccebd15-c984-4e57-aef8-329128bb254f admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:56.753917 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84422c3f-e06f-477d-82e7-0e2fd300af4d None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:56.754442 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84422c3f-e06f-477d-82e7-0e2fd300af4d None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:56.755449 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84422c3f-e06f-477d-82e7-0e2fd300af4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:56.764962 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e0b630-9828-4610-a2a0-13c62bba73f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:56.765652 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e0b630-9828-4610-a2a0-13c62bba73f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:56.766115 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e0b630-9828-4610-a2a0-13c62bba73f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:56.766441 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e0b630-9828-4610-a2a0-13c62bba73f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:56.767374 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e0b630-9828-4610-a2a0-13c62bba73f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:56.769485 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e0b630-9828-4610-a2a0-13c62bba73f6 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:56.789322 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-84422c3f-e06f-477d-82e7-0e2fd300af4d None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:08:56.790865 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-84422c3f-e06f-477d-82e7-0e2fd300af4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:56.800005 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 918/1800] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:08:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:08:56.808003 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f102c2-51d0-491a-9fac-d8f719da2b0a None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:56.808374 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f102c2-51d0-491a-9fac-d8f719da2b0a None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:56.808717 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f102c2-51d0-491a-9fac-d8f719da2b0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:56.832296 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-32f102c2-51d0-491a-9fac-d8f719da2b0a None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:08:56.833474 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-32f102c2-51d0-491a-9fac-d8f719da2b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:56.843628 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 919/1801] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:08:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:08:56.847863 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f102c2-51d0-491a-9fac-d8f719da2b0a None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:56.859983 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f102c2-51d0-491a-9fac-d8f719da2b0a None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:56.866878 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a7416f-c634-445a-9dec-d568021b09f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:56.867901 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a7416f-c634-445a-9dec-d568021b09f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:56.868820 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a7416f-c634-445a-9dec-d568021b09f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:56.868820 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a7416f-c634-445a-9dec-d568021b09f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:56.869688 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a7416f-c634-445a-9dec-d568021b09f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 03:08:56.870365 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 883/1802] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:08:56 2022] POST /identity/v3/projects => generated 365 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:08:56.873615 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a7416f-c634-445a-9dec-d568021b09f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:56.874475 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e0b630-9828-4610-a2a0-13c62bba73f6 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:56.875916 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a7416f-c634-445a-9dec-d568021b09f4 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:56.877049 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-87e0b630-9828-4610-a2a0-13c62bba73f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:56.896219 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 920/1803] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Oct 14 03:08:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 03:08:56.898938 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a7416f-c634-445a-9dec-d568021b09f4 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:56.912446 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a7416f-c634-445a-9dec-d568021b09f4 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:56.981281 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e0b630-9828-4610-a2a0-13c62bba73f6 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:56.988050 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156aeb4f-9b4b-468f-83a1-cc82c799ea11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:56.989110 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156aeb4f-9b4b-468f-83a1-cc82c799ea11 admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:56.989110 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156aeb4f-9b4b-468f-83a1-cc82c799ea11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:56.989331 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156aeb4f-9b4b-468f-83a1-cc82c799ea11 admin admin] PATH_INFO: `/v3/users` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:56.990046 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156aeb4f-9b4b-468f-83a1-cc82c799ea11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:56.992178 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156aeb4f-9b4b-468f-83a1-cc82c799ea11 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:57.034493 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caef5fd-1b6b-441c-a5b2-8290b543136b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:57.035148 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caef5fd-1b6b-441c-a5b2-8290b543136b admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:57.035452 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caef5fd-1b6b-441c-a5b2-8290b543136b admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:57.035713 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caef5fd-1b6b-441c-a5b2-8290b543136b admin admin] PATH_INFO: `/v3/projects` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:57.036383 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caef5fd-1b6b-441c-a5b2-8290b543136b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:57.039271 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caef5fd-1b6b-441c-a5b2-8290b543136b admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:57.135604 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 921/1804] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:08:56 2022] POST /identity/v3/projects => generated 383 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:08:57.135766 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 884/1805] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:08:56 2022] POST /identity/v3/users => generated 283 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:08:57.135864 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf64555-4229-4eb3-b48f-8ea9e8d3c464 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:57.135864 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf64555-4229-4eb3-b48f-8ea9e8d3c464 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:57.136022 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf64555-4229-4eb3-b48f-8ea9e8d3c464 None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:57.136927 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156aeb4f-9b4b-468f-83a1-cc82c799ea11 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:57.137218 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 922/1806] 10.222.0.39 () {54 vars in 921 bytes} [Fri Oct 14 03:08:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 03:08:57.137936 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf64555-4229-4eb3-b48f-8ea9e8d3c464 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:57.150774 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156aeb4f-9b4b-468f-83a1-cc82c799ea11 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:57.216058 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf64555-4229-4eb3-b48f-8ea9e8d3c464 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:57.219243 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:57.221491 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633eaa8a-2562-41bd-b1ad-e9f9195c445a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:57.222159 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633eaa8a-2562-41bd-b1ad-e9f9195c445a admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:57.222558 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633eaa8a-2562-41bd-b1ad-e9f9195c445a admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:57.222838 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633eaa8a-2562-41bd-b1ad-e9f9195c445a admin admin] PATH_INFO: `/v3/users` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:57.224557 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633eaa8a-2562-41bd-b1ad-e9f9195c445a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:57.225653 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633eaa8a-2562-41bd-b1ad-e9f9195c445a admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:57.233107 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d6b7a2-df22-4c55-b7c1-b9492e4b367e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:57.233984 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d6b7a2-df22-4c55-b7c1-b9492e4b367e admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:57.234579 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d6b7a2-df22-4c55-b7c1-b9492e4b367e admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:57.234860 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d6b7a2-df22-4c55-b7c1-b9492e4b367e admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:57.236017 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d6b7a2-df22-4c55-b7c1-b9492e4b367e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:57.238379 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d6b7a2-df22-4c55-b7c1-b9492e4b367e admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:57.256910 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 885/1807] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:08:57 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:08:57.259137 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab168ff5-3a22-4818-9c68-7443d45fbb34 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:57.259445 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab168ff5-3a22-4818-9c68-7443d45fbb34 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:57.259726 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab168ff5-3a22-4818-9c68-7443d45fbb34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:57.286446 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-ab168ff5-3a22-4818-9c68-7443d45fbb34 None None] MFA Rules not processed for user `922ef7811aed450baff7f936011b0af1`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:08:57.287948 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-ab168ff5-3a22-4818-9c68-7443d45fbb34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:57.336868 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 923/1808] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:08:57 2022] POST /identity/v3/users => generated 292 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:08:57.338137 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633eaa8a-2562-41bd-b1ad-e9f9195c445a admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:57.349697 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633eaa8a-2562-41bd-b1ad-e9f9195c445a admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:57.356270 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf10589-7e4c-4b35-ae5a-cb9307ca205e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:57.356871 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf10589-7e4c-4b35-ae5a-cb9307ca205e admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:57.357149 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf10589-7e4c-4b35-ae5a-cb9307ca205e admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:57.357422 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf10589-7e4c-4b35-ae5a-cb9307ca205e admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:57.359241 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf10589-7e4c-4b35-ae5a-cb9307ca205e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:57.360202 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf10589-7e4c-4b35-ae5a-cb9307ca205e admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:57.372063 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 924/1809] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:08:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:08:57.372655 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf10589-7e4c-4b35-ae5a-cb9307ca205e admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:57.374510 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-fbf10589-7e4c-4b35-ae5a-cb9307ca205e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:57.385429 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 886/1810] 10.222.0.39 () {58 vars in 1018 bytes} [Fri Oct 14 03:08:57 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:08:57.386383 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab168ff5-3a22-4818-9c68-7443d45fbb34 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:57.388731 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf10589-7e4c-4b35-ae5a-cb9307ca205e admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:57.393713 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3d8c7a-e2a6-4f29-852d-dc0782f2ab19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:57.395408 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3d8c7a-e2a6-4f29-852d-dc0782f2ab19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:57.395408 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3d8c7a-e2a6-4f29-852d-dc0782f2ab19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:57.395408 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3d8c7a-e2a6-4f29-852d-dc0782f2ab19 admin admin] PATH_INFO: `/v3/users/7517af640df3402783a38befbf3d3908` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:57.396213 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3d8c7a-e2a6-4f29-852d-dc0782f2ab19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7517af640df3402783a38befbf3d3908)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:57.398755 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3d8c7a-e2a6-4f29-852d-dc0782f2ab19 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:57.401342 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:57.408013 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab168ff5-3a22-4818-9c68-7443d45fbb34 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:57.414278 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2f8378-a967-4ed5-888f-1aabf1d90b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:57.414898 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2f8378-a967-4ed5-888f-1aabf1d90b20 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:57.415179 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2f8378-a967-4ed5-888f-1aabf1d90b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:57.415505 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2f8378-a967-4ed5-888f-1aabf1d90b20 admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:57.416108 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2f8378-a967-4ed5-888f-1aabf1d90b20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:57.418060 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2f8378-a967-4ed5-888f-1aabf1d90b20 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:57.429587 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 887/1811] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:08:57 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:08:57.430460 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2f8378-a967-4ed5-888f-1aabf1d90b20 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:57.440769 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2f8378-a967-4ed5-888f-1aabf1d90b20 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:57.447021 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ff8b5d-4dc4-4afa-85ce-b978b5631c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:57.447699 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ff8b5d-4dc4-4afa-85ce-b978b5631c15 admin admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:57.447976 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ff8b5d-4dc4-4afa-85ce-b978b5631c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:57.448247 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ff8b5d-4dc4-4afa-85ce-b978b5631c15 admin admin] PATH_INFO: `/v3/projects/73cab3822d4d43c2a1c6ccac1e2d984c/users/a547dffc3e0147b9a91ec0f40ca08ccb/roles/9bd0470b544240209a420f640c79c06c` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:57.462630 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ff8b5d-4dc4-4afa-85ce-b978b5631c15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73cab3822d4d43c2a1c6ccac1e2d984c, user_id=a547dffc3e0147b9a91ec0f40ca08ccb, role_id=9bd0470b544240209a420f640c79c06c)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:57.465259 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ff8b5d-4dc4-4afa-85ce-b978b5631c15 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:57.567692 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 888/1812] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:08:57 2022] PUT /identity/v3/projects/73cab3822d4d43c2a1c6ccac1e2d984c/users/a547dffc3e0147b9a91ec0f40ca08ccb/roles/9bd0470b544240209a420f640c79c06c => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:08:57.568322 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ff8b5d-4dc4-4afa-85ce-b978b5631c15 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:57.570514 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-38ff8b5d-4dc4-4afa-85ce-b978b5631c15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:57.610258 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-5f3d8c7a-e2a6-4f29-852d-dc0782f2ab19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7517af640df3402783a38befbf3d3908'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:57.610949 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-5f3d8c7a-e2a6-4f29-852d-dc0782f2ab19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7517af640df3402783a38befbf3d3908'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:57.645670 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ff8b5d-4dc4-4afa-85ce-b978b5631c15 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:57.651187 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84b711f-7b42-4db5-9ed4-db37c9904dd6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '922ef7811aed450baff7f936011b0af1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:57.651806 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84b711f-7b42-4db5-9ed4-db37c9904dd6 service placement] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:57.652094 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84b711f-7b42-4db5-9ed4-db37c9904dd6 service placement] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:57.652357 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84b711f-7b42-4db5-9ed4-db37c9904dd6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:57.654341 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-d84b711f-7b42-4db5-9ed4-db37c9904dd6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:57.668963 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84b711f-7b42-4db5-9ed4-db37c9904dd6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:57.669921 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84b711f-7b42-4db5-9ed4-db37c9904dd6 service placement] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:57.681136 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 889/1813] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:08:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:08:57.682013 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84b711f-7b42-4db5-9ed4-db37c9904dd6 service placement] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:57.683974 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-5f3d8c7a-e2a6-4f29-852d-dc0782f2ab19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7517af640df3402783a38befbf3d3908'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:57.684601 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-5f3d8c7a-e2a6-4f29-852d-dc0782f2ab19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7517af640df3402783a38befbf3d3908'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:57.690609 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-d84b711f-7b42-4db5-9ed4-db37c9904dd6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:57.693290 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-5f3d8c7a-e2a6-4f29-852d-dc0782f2ab19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7517af640df3402783a38befbf3d3908'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:57.712878 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 925/1814] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:08:57 2022] DELETE /identity/v3/users/7517af640df3402783a38befbf3d3908 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:08:57.718468 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439aed40-4c0a-4a80-9c4d-d3006b0a322b None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:57.718468 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439aed40-4c0a-4a80-9c4d-d3006b0a322b None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:57.718468 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439aed40-4c0a-4a80-9c4d-d3006b0a322b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:57.749278 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:57.751205 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-439aed40-4c0a-4a80-9c4d-d3006b0a322b None None] MFA Rules not processed for user `a547dffc3e0147b9a91ec0f40ca08ccb`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:08:57.752465 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-439aed40-4c0a-4a80-9c4d-d3006b0a322b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:57.781076 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84b711f-7b42-4db5-9ed4-db37c9904dd6 service placement] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:57.840491 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 926/1815] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:08:57 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:08:57.854467 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439aed40-4c0a-4a80-9c4d-d3006b0a322b None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:57.857199 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-439aed40-4c0a-4a80-9c4d-d3006b0a322b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:57.861030 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b42ca5f-f9c4-4e0c-a172-f083d5971e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:57.861778 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b42ca5f-f9c4-4e0c-a172-f083d5971e09 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:57.862722 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b42ca5f-f9c4-4e0c-a172-f083d5971e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:57.863006 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b42ca5f-f9c4-4e0c-a172-f083d5971e09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:57.863697 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b42ca5f-f9c4-4e0c-a172-f083d5971e09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:57.866008 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b42ca5f-f9c4-4e0c-a172-f083d5971e09 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:57.877996 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 890/1816] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:08:57 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:08:57.882362 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b42ca5f-f9c4-4e0c-a172-f083d5971e09 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:57.903067 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b42ca5f-f9c4-4e0c-a172-f083d5971e09 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:57.907944 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3042e7-f21d-45db-a254-c7dab25b5f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:57.908628 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3042e7-f21d-45db-a254-c7dab25b5f8c admin admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:57.908871 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3042e7-f21d-45db-a254-c7dab25b5f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:57.909446 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3042e7-f21d-45db-a254-c7dab25b5f8c admin admin] PATH_INFO: `/v3/projects/ca923908ecd6487faa038ff7e7da317b/users/fb1ec5ec438d46f7baf506207dd0eaec/roles/9bd0470b544240209a420f640c79c06c` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:57.925560 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3042e7-f21d-45db-a254-c7dab25b5f8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca923908ecd6487faa038ff7e7da317b, user_id=fb1ec5ec438d46f7baf506207dd0eaec, role_id=9bd0470b544240209a420f640c79c06c)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:57.928405 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3042e7-f21d-45db-a254-c7dab25b5f8c admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:57.929909 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439aed40-4c0a-4a80-9c4d-d3006b0a322b None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:57.934431 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf881c1-e72e-4ddc-af23-a1f2f715e7ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab599d98b0624a2e89090312bcab713f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:57.935160 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf881c1-e72e-4ddc-af23-a1f2f715e7ab service cinder] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:57.935504 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf881c1-e72e-4ddc-af23-a1f2f715e7ab service cinder] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:57.935837 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf881c1-e72e-4ddc-af23-a1f2f715e7ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:57.944623 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf881c1-e72e-4ddc-af23-a1f2f715e7ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:57.946088 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf881c1-e72e-4ddc-af23-a1f2f715e7ab service cinder] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:57.955645 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 927/1817] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:08:57 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 03:08:58.023238 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 891/1818] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:08:57 2022] PUT /identity/v3/projects/ca923908ecd6487faa038ff7e7da317b/users/fb1ec5ec438d46f7baf506207dd0eaec/roles/9bd0470b544240209a420f640c79c06c => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 03:08:58.024615 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf881c1-e72e-4ddc-af23-a1f2f715e7ab service cinder] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:58.028938 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-baf881c1-e72e-4ddc-af23-a1f2f715e7ab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:58.037954 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c93df-a83f-4069-af4f-eff150854c37 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:58.038899 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c93df-a83f-4069-af4f-eff150854c37 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:58.039179 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c93df-a83f-4069-af4f-eff150854c37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:58.055049 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf881c1-e72e-4ddc-af23-a1f2f715e7ab service cinder] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:58.065878 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:58.084881 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-c20c93df-a83f-4069-af4f-eff150854c37 None None] MFA Rules not processed for user `fb1ec5ec438d46f7baf506207dd0eaec`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:08:58.086139 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-c20c93df-a83f-4069-af4f-eff150854c37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:58.121808 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc4d7cc-93ba-48b0-9aed-1d8b69883853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:58.122475 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc4d7cc-93ba-48b0-9aed-1d8b69883853 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:58.122893 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc4d7cc-93ba-48b0-9aed-1d8b69883853 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:58.123123 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc4d7cc-93ba-48b0-9aed-1d8b69883853 admin admin] PATH_INFO: `/v3/projects/5cd113c7d65543b2bac8ef368467c092` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:58.124419 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc4d7cc-93ba-48b0-9aed-1d8b69883853 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cd113c7d65543b2bac8ef368467c092)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:58.126635 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc4d7cc-93ba-48b0-9aed-1d8b69883853 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:58.153601 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 892/1819] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:08:58 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 03:08:58.165609 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20c93df-a83f-4069-af4f-eff150854c37 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:58.167004 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-c20c93df-a83f-4069-af4f-eff150854c37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:58.214874 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:58.225493 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20c93df-a83f-4069-af4f-eff150854c37 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:58.234372 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-6cc4d7cc-93ba-48b0-9aed-1d8b69883853 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cd113c7d65543b2bac8ef368467c092', 'request_id': 'req-6cc4d7cc-93ba-48b0-9aed-1d8b69883853'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:58.240488 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-6cc4d7cc-93ba-48b0-9aed-1d8b69883853 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cd113c7d65543b2bac8ef368467c092', 'request_id': 'req-6cc4d7cc-93ba-48b0-9aed-1d8b69883853'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:58.274317 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc7bc0b-879d-4d06-9842-66015da8654a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab599d98b0624a2e89090312bcab713f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:58.274927 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc7bc0b-879d-4d06-9842-66015da8654a service cinder] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:58.275213 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc7bc0b-879d-4d06-9842-66015da8654a service cinder] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:58.275482 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc7bc0b-879d-4d06-9842-66015da8654a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:58.323147 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-6cc4d7cc-93ba-48b0-9aed-1d8b69883853 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cd113c7d65543b2bac8ef368467c092', 'request_id': 'req-6cc4d7cc-93ba-48b0-9aed-1d8b69883853'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:58.333647 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc7bc0b-879d-4d06-9842-66015da8654a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:58.335651 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc7bc0b-879d-4d06-9842-66015da8654a service cinder] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:58.338280 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-6cc4d7cc-93ba-48b0-9aed-1d8b69883853 admin admin] The token cache is being invalidate because project 5cd113c7d65543b2bac8ef368467c092 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81517) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:08:58.338674 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-6cc4d7cc-93ba-48b0-9aed-1d8b69883853 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:58.339669 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-6cc4d7cc-93ba-48b0-9aed-1d8b69883853 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:58.341956 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 928/1820] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:08:58 2022] DELETE /identity/v3/projects/5cd113c7d65543b2bac8ef368467c092 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:08:58.347058 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 893/1821] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:08:58 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 03:08:58.347627 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b88eb03-bcd6-4f9a-9606-b27ebbe76ceb None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:58.347940 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b88eb03-bcd6-4f9a-9606-b27ebbe76ceb None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:58.348215 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b88eb03-bcd6-4f9a-9606-b27ebbe76ceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:58.367635 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc7bc0b-879d-4d06-9842-66015da8654a service cinder] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:58.370589 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-9b88eb03-bcd6-4f9a-9606-b27ebbe76ceb None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:08:58.371805 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-9b88eb03-bcd6-4f9a-9606-b27ebbe76ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:58.436824 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 929/1822] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:08:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:08:58.438479 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc7bc0b-879d-4d06-9842-66015da8654a service cinder] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:58.443705 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9680c39e-63d1-4e6a-90d5-7b09a57d3a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:58.444736 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549a79c2-72ab-47d2-96a0-e179743dbdaa None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:58.444855 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9680c39e-63d1-4e6a-90d5-7b09a57d3a28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:58.444934 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549a79c2-72ab-47d2-96a0-e179743dbdaa None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:58.445019 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9680c39e-63d1-4e6a-90d5-7b09a57d3a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:58.445113 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549a79c2-72ab-47d2-96a0-e179743dbdaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:58.445328 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9680c39e-63d1-4e6a-90d5-7b09a57d3a28 admin admin] PATH_INFO: `/v3/users/fb1ec5ec438d46f7baf506207dd0eaec` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:58.446409 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9680c39e-63d1-4e6a-90d5-7b09a57d3a28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb1ec5ec438d46f7baf506207dd0eaec)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:58.448417 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9680c39e-63d1-4e6a-90d5-7b09a57d3a28 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:58.469066 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-549a79c2-72ab-47d2-96a0-e179743dbdaa None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:08:58.470403 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-549a79c2-72ab-47d2-96a0-e179743dbdaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:58.479608 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 930/1823] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:08:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 03:08:58.484078 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549a79c2-72ab-47d2-96a0-e179743dbdaa None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:58.493575 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549a79c2-72ab-47d2-96a0-e179743dbdaa None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:58.498485 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab35fa0-8b30-43f5-a705-fb5294bb45c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:58.499543 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab35fa0-8b30-43f5-a705-fb5294bb45c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:58.499543 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab35fa0-8b30-43f5-a705-fb5294bb45c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:58.499744 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab35fa0-8b30-43f5-a705-fb5294bb45c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:58.500393 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab35fa0-8b30-43f5-a705-fb5294bb45c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 03:08:58.500700 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab35fa0-8b30-43f5-a705-fb5294bb45c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:58.502367 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab35fa0-8b30-43f5-a705-fb5294bb45c2 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:58.514345 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 931/1824] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Oct 14 03:08:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 03:08:58.518515 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab35fa0-8b30-43f5-a705-fb5294bb45c2 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:58.535939 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab35fa0-8b30-43f5-a705-fb5294bb45c2 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:58.542335 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec58d93-06c0-4e3d-b9b5-9381d47e3fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:58.542946 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec58d93-06c0-4e3d-b9b5-9381d47e3fa0 admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:58.545821 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec58d93-06c0-4e3d-b9b5-9381d47e3fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:58.546178 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec58d93-06c0-4e3d-b9b5-9381d47e3fa0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:58.547483 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec58d93-06c0-4e3d-b9b5-9381d47e3fa0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:58.549733 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec58d93-06c0-4e3d-b9b5-9381d47e3fa0 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:58.663704 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 932/1825] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:08:58 2022] POST /identity/v3/projects => generated 357 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 03:08:58.667573 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec58d93-06c0-4e3d-b9b5-9381d47e3fa0 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:58.672881 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:58.674799 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-9680c39e-63d1-4e6a-90d5-7b09a57d3a28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb1ec5ec438d46f7baf506207dd0eaec'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:58.684704 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-9680c39e-63d1-4e6a-90d5-7b09a57d3a28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb1ec5ec438d46f7baf506207dd0eaec'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:58.685813 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:58.701512 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-9680c39e-63d1-4e6a-90d5-7b09a57d3a28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb1ec5ec438d46f7baf506207dd0eaec'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:58.702726 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-9680c39e-63d1-4e6a-90d5-7b09a57d3a28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb1ec5ec438d46f7baf506207dd0eaec'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:58.738370 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec58d93-06c0-4e3d-b9b5-9381d47e3fa0 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:58.743171 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd0d1a7-dadb-4b34-884c-151fda525eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:58.743773 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd0d1a7-dadb-4b34-884c-151fda525eb6 admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:58.744059 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd0d1a7-dadb-4b34-884c-151fda525eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:58.744337 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd0d1a7-dadb-4b34-884c-151fda525eb6 admin admin] PATH_INFO: `/v3/users` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:58.745095 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd0d1a7-dadb-4b34-884c-151fda525eb6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:58.747070 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd0d1a7-dadb-4b34-884c-151fda525eb6 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:58.781436 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-9680c39e-63d1-4e6a-90d5-7b09a57d3a28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb1ec5ec438d46f7baf506207dd0eaec'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:58.787635 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 894/1826] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:08:58 2022] DELETE /identity/v3/users/fb1ec5ec438d46f7baf506207dd0eaec => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 03:08:58.791942 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06efa49f-739d-4834-ac48-f9e233cb7249 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:58.792226 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06efa49f-739d-4834-ac48-f9e233cb7249 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:58.792500 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06efa49f-739d-4834-ac48-f9e233cb7249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:58.811273 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-06efa49f-739d-4834-ac48-f9e233cb7249 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:08:58.812554 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-06efa49f-739d-4834-ac48-f9e233cb7249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:58.867671 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 933/1827] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:08:58 2022] POST /identity/v3/users => generated 279 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 03:08:58.871289 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd0d1a7-dadb-4b34-884c-151fda525eb6 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:58.872791 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 895/1828] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:08:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 03:08:58.875868 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-dcd0d1a7-dadb-4b34-884c-151fda525eb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:58.882934 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06efa49f-739d-4834-ac48-f9e233cb7249 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:58.884419 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-06efa49f-739d-4834-ac48-f9e233cb7249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:58.887153 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd0d1a7-dadb-4b34-884c-151fda525eb6 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:58.892177 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0f2664-acba-4309-a0a0-334826464e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:58.892786 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0f2664-acba-4309-a0a0-334826464e57 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:58.893074 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0f2664-acba-4309-a0a0-334826464e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:58.893354 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0f2664-acba-4309-a0a0-334826464e57 admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:58.894142 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0f2664-acba-4309-a0a0-334826464e57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:58.895752 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0f2664-acba-4309-a0a0-334826464e57 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:58.907338 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 934/1829] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:08:58 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 03:08:58.911231 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0f2664-acba-4309-a0a0-334826464e57 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:58.921233 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0f2664-acba-4309-a0a0-334826464e57 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:58.926185 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c01164-2320-48b8-88ec-b2f0d2d3da1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:58.926737 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c01164-2320-48b8-88ec-b2f0d2d3da1c admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:58.927137 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c01164-2320-48b8-88ec-b2f0d2d3da1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:58.927419 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c01164-2320-48b8-88ec-b2f0d2d3da1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:58.928049 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c01164-2320-48b8-88ec-b2f0d2d3da1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:58.930194 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c01164-2320-48b8-88ec-b2f0d2d3da1c admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:58.940234 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06efa49f-739d-4834-ac48-f9e233cb7249 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:58.941400 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 935/1830] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:08:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 03:08:58.943933 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e86533f-8067-428e-b82b-a2e7b10fd128 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:58.945126 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e86533f-8067-428e-b82b-a2e7b10fd128 service neutron] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:58.945374 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e86533f-8067-428e-b82b-a2e7b10fd128 service neutron] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:58.945648 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e86533f-8067-428e-b82b-a2e7b10fd128 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:58.946096 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c01164-2320-48b8-88ec-b2f0d2d3da1c admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:58.954987 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e86533f-8067-428e-b82b-a2e7b10fd128 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:58.956415 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c01164-2320-48b8-88ec-b2f0d2d3da1c admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:58.961328 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e86533f-8067-428e-b82b-a2e7b10fd128 service neutron] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:58.961486 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689c92ea-9e04-426d-b84e-bf745863ea90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:58.961617 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689c92ea-9e04-426d-b84e-bf745863ea90 admin admin] REQUEST_METHOD: `PUT` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:58.961909 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689c92ea-9e04-426d-b84e-bf745863ea90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:58.962186 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689c92ea-9e04-426d-b84e-bf745863ea90 admin admin] PATH_INFO: `/v3/projects/087d77718a3d414dace397802d70873f/users/f3d01587bbde467f8e26dfb399bd4a32/roles/9bd0470b544240209a420f640c79c06c` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:58.966182 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 896/1831] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:08:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:08:58.977541 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689c92ea-9e04-426d-b84e-bf745863ea90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=087d77718a3d414dace397802d70873f, user_id=f3d01587bbde467f8e26dfb399bd4a32, role_id=9bd0470b544240209a420f640c79c06c)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:58.980388 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689c92ea-9e04-426d-b84e-bf745863ea90 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:59.084396 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 936/1832] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:08:58 2022] PUT /identity/v3/projects/087d77718a3d414dace397802d70873f/users/f3d01587bbde467f8e26dfb399bd4a32/roles/9bd0470b544240209a420f640c79c06c => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 03:08:59.094774 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876d3c5d-967a-432a-a075-c49e86a34e87 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:59.095074 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876d3c5d-967a-432a-a075-c49e86a34e87 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:59.095338 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876d3c5d-967a-432a-a075-c49e86a34e87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:59.129414 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-876d3c5d-967a-432a-a075-c49e86a34e87 None None] MFA Rules not processed for user `f3d01587bbde467f8e26dfb399bd4a32`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:08:59.130134 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-876d3c5d-967a-432a-a075-c49e86a34e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:59.198580 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 897/1833] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:08:59 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:08:59.203047 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689c92ea-9e04-426d-b84e-bf745863ea90 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:59.273447 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689c92ea-9e04-426d-b84e-bf745863ea90 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:59.278155 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4de36fd-558e-47c4-abef-bf0d913fc445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:59.278764 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4de36fd-558e-47c4-abef-bf0d913fc445 admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:59.279046 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4de36fd-558e-47c4-abef-bf0d913fc445 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:59.279322 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4de36fd-558e-47c4-abef-bf0d913fc445 admin admin] PATH_INFO: `/v3/projects` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:59.279960 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4de36fd-558e-47c4-abef-bf0d913fc445 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:59.281903 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4de36fd-558e-47c4-abef-bf0d913fc445 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:59.371643 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 937/1834] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:08:59 2022] POST /identity/v3/projects => generated 357 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:08:59.375794 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876d3c5d-967a-432a-a075-c49e86a34e87 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:59.386681 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876d3c5d-967a-432a-a075-c49e86a34e87 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:59.390234 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:59.442179 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e804091e-c355-4917-8306-1b8e77e33b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:59.442830 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e804091e-c355-4917-8306-1b8e77e33b12 admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:59.443014 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e804091e-c355-4917-8306-1b8e77e33b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:59.443286 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e804091e-c355-4917-8306-1b8e77e33b12 admin admin] PATH_INFO: `/v3/users` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:59.443916 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e804091e-c355-4917-8306-1b8e77e33b12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:59.445868 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e804091e-c355-4917-8306-1b8e77e33b12 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:59.457232 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4de36fd-558e-47c4-abef-bf0d913fc445 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:59.459886 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-d4de36fd-558e-47c4-abef-bf0d913fc445 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:59.474520 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4de36fd-558e-47c4-abef-bf0d913fc445 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:59.485035 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6181511b-b530-4dc1-a884-2c82b653ddcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:59.485035 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6181511b-b530-4dc1-a884-2c82b653ddcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:59.485035 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6181511b-b530-4dc1-a884-2c82b653ddcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:59.485035 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6181511b-b530-4dc1-a884-2c82b653ddcd admin admin] PATH_INFO: `/v3/projects/ca923908ecd6487faa038ff7e7da317b` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:59.485446 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6181511b-b530-4dc1-a884-2c82b653ddcd admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca923908ecd6487faa038ff7e7da317b)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:59.485446 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6181511b-b530-4dc1-a884-2c82b653ddcd admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:59.524081 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 898/1835] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:08:59 2022] POST /identity/v3/users => generated 285 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:08:59.528482 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e804091e-c355-4917-8306-1b8e77e33b12 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:59.539672 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e804091e-c355-4917-8306-1b8e77e33b12 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:59.544591 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37965b2c-a485-489f-901d-5d58cb00d6e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:59.545338 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37965b2c-a485-489f-901d-5d58cb00d6e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:59.545625 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37965b2c-a485-489f-901d-5d58cb00d6e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:59.545943 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37965b2c-a485-489f-901d-5d58cb00d6e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:59.546587 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37965b2c-a485-489f-901d-5d58cb00d6e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:59.548185 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37965b2c-a485-489f-901d-5d58cb00d6e5 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:59.558818 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 899/1836] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:08:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 03:08:59.562861 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37965b2c-a485-489f-901d-5d58cb00d6e5 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:59.572447 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37965b2c-a485-489f-901d-5d58cb00d6e5 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:59.577238 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ba2551-c84e-4a71-8510-8c38b7b7c7d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:59.577889 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ba2551-c84e-4a71-8510-8c38b7b7c7d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:59.578180 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ba2551-c84e-4a71-8510-8c38b7b7c7d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:59.578462 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ba2551-c84e-4a71-8510-8c38b7b7c7d8 admin admin] PATH_INFO: `/v3/projects/36f2b1a55312424d9246dc70f6517e84/users/8b3e86586030436c97bb45f1cb056348/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:59.582985 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:08:59.593611 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ba2551-c84e-4a71-8510-8c38b7b7c7d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36f2b1a55312424d9246dc70f6517e84, user_id=8b3e86586030436c97bb45f1cb056348, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:59.596306 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ba2551-c84e-4a71-8510-8c38b7b7c7d8 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:59.599566 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-6181511b-b530-4dc1-a884-2c82b653ddcd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca923908ecd6487faa038ff7e7da317b', 'request_id': 'req-6181511b-b530-4dc1-a884-2c82b653ddcd'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:59.610287 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-6181511b-b530-4dc1-a884-2c82b653ddcd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca923908ecd6487faa038ff7e7da317b', 'request_id': 'req-6181511b-b530-4dc1-a884-2c82b653ddcd'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:59.657242 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 900/1837] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:08:59 2022] PUT /identity/v3/projects/36f2b1a55312424d9246dc70f6517e84/users/8b3e86586030436c97bb45f1cb056348/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 03:08:59.661711 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ba2551-c84e-4a71-8510-8c38b7b7c7d8 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:59.686474 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-6181511b-b530-4dc1-a884-2c82b653ddcd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca923908ecd6487faa038ff7e7da317b', 'request_id': 'req-6181511b-b530-4dc1-a884-2c82b653ddcd'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:59.707296 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-6181511b-b530-4dc1-a884-2c82b653ddcd admin admin] The token cache is being invalidate because project ca923908ecd6487faa038ff7e7da317b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81517) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:08:59.707593 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-6181511b-b530-4dc1-a884-2c82b653ddcd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:59.709572 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-6181511b-b530-4dc1-a884-2c82b653ddcd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:08:59.712913 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 938/1838] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:08:59 2022] DELETE /identity/v3/projects/ca923908ecd6487faa038ff7e7da317b => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 03:08:59.719539 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd497ee-ee77-4d44-8c3d-a4cd328cc2b7 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:59.719688 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd497ee-ee77-4d44-8c3d-a4cd328cc2b7 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:59.719966 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd497ee-ee77-4d44-8c3d-a4cd328cc2b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:59.748747 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-bbd497ee-ee77-4d44-8c3d-a4cd328cc2b7 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:08:59.750006 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-bbd497ee-ee77-4d44-8c3d-a4cd328cc2b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:59.782682 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ba2551-c84e-4a71-8510-8c38b7b7c7d8 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:59.795208 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee39749f-de77-404a-8354-fcead4633653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:59.797743 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee39749f-de77-404a-8354-fcead4633653 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:59.802812 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee39749f-de77-404a-8354-fcead4633653 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:59.802812 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee39749f-de77-404a-8354-fcead4633653 admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:59.802812 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee39749f-de77-404a-8354-fcead4633653 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:59.802812 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee39749f-de77-404a-8354-fcead4633653 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:59.815634 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 901/1839] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:08:59 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:08:59.820790 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee39749f-de77-404a-8354-fcead4633653 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:59.822561 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-ee39749f-de77-404a-8354-fcead4633653 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:59.843062 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee39749f-de77-404a-8354-fcead4633653 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:59.849061 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622ab56e-1b9f-45e9-a4c4-a92daf69ea80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:59.849784 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ab56e-1b9f-45e9-a4c4-a92daf69ea80 admin admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:59.850075 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ab56e-1b9f-45e9-a4c4-a92daf69ea80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:59.850361 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ab56e-1b9f-45e9-a4c4-a92daf69ea80 admin admin] PATH_INFO: `/v3/projects/36f2b1a55312424d9246dc70f6517e84/users/8b3e86586030436c97bb45f1cb056348/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:59.852861 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622ab56e-1b9f-45e9-a4c4-a92daf69ea80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36f2b1a55312424d9246dc70f6517e84, user_id=8b3e86586030436c97bb45f1cb056348, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:59.855880 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622ab56e-1b9f-45e9-a4c4-a92daf69ea80 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:08:59.857152 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 939/1840] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:08:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 03:08:59.865435 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f5570c-e27b-445e-babe-12cbcd89765a None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:59.866924 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f5570c-e27b-445e-babe-12cbcd89765a None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:59.867568 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f5570c-e27b-445e-babe-12cbcd89765a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:59.876015 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 902/1841] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:08:59 2022] PUT /identity/v3/projects/36f2b1a55312424d9246dc70f6517e84/users/8b3e86586030436c97bb45f1cb056348/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:08:59.888348 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0476844a-6cf7-4f6a-8775-3ca34f2c8635 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:59.888779 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0476844a-6cf7-4f6a-8775-3ca34f2c8635 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:59.889437 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0476844a-6cf7-4f6a-8775-3ca34f2c8635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:59.898180 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-a7f5570c-e27b-445e-babe-12cbcd89765a None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:08:59.899472 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-a7f5570c-e27b-445e-babe-12cbcd89765a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:59.922855 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-0476844a-6cf7-4f6a-8775-3ca34f2c8635 None None] MFA Rules not processed for user `8b3e86586030436c97bb45f1cb056348`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:08:59.924070 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-0476844a-6cf7-4f6a-8775-3ca34f2c8635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:08:59.974678 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 940/1842] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:08:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:08:59.975731 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f5570c-e27b-445e-babe-12cbcd89765a None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:08:59.985779 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f5570c-e27b-445e-babe-12cbcd89765a None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:08:59.991901 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9ff187-14e7-47a6-97ea-b049576fbff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:08:59.992586 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9ff187-14e7-47a6-97ea-b049576fbff4 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:08:59.992782 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9ff187-14e7-47a6-97ea-b049576fbff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:08:59.993058 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9ff187-14e7-47a6-97ea-b049576fbff4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:08:59.993714 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9ff187-14e7-47a6-97ea-b049576fbff4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 03:08:59.994053 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9ff187-14e7-47a6-97ea-b049576fbff4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:08:59.995780 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9ff187-14e7-47a6-97ea-b049576fbff4 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:00.008661 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 941/1843] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Oct 14 03:08:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 03:09:00.014862 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9ff187-14e7-47a6-97ea-b049576fbff4 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:00.026103 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9ff187-14e7-47a6-97ea-b049576fbff4 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:00.032998 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7568f927-59d5-4c7d-9391-490e4645dba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:00.033651 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7568f927-59d5-4c7d-9391-490e4645dba1 admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:00.033984 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7568f927-59d5-4c7d-9391-490e4645dba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:00.034317 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7568f927-59d5-4c7d-9391-490e4645dba1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:00.035055 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7568f927-59d5-4c7d-9391-490e4645dba1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:00.037017 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7568f927-59d5-4c7d-9391-490e4645dba1 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:00.044145 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 903/1844] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:08:59 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 03:09:00.056210 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0476844a-6cf7-4f6a-8775-3ca34f2c8635 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:00.060437 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-0476844a-6cf7-4f6a-8775-3ca34f2c8635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:09:00.074126 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:09:00.115193 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 942/1845] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:09:00 2022] POST /identity/v3/projects => generated 377 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 03:09:00.119326 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7568f927-59d5-4c7d-9391-490e4645dba1 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:00.125217 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0476844a-6cf7-4f6a-8775-3ca34f2c8635 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:00.184284 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdfe6f7-4020-4a70-8c2a-bd4cb74611bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab599d98b0624a2e89090312bcab713f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:00.184962 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdfe6f7-4020-4a70-8c2a-bd4cb74611bc service cinder] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:00.185351 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdfe6f7-4020-4a70-8c2a-bd4cb74611bc service cinder] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:00.185629 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdfe6f7-4020-4a70-8c2a-bd4cb74611bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:00.188323 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7568f927-59d5-4c7d-9391-490e4645dba1 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:00.193134 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6691a2a-2a23-4999-8360-7fd815dfb2c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:00.193665 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6691a2a-2a23-4999-8360-7fd815dfb2c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:00.193980 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6691a2a-2a23-4999-8360-7fd815dfb2c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:00.194258 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6691a2a-2a23-4999-8360-7fd815dfb2c5 admin admin] PATH_INFO: `/v3/users` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:00.195518 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6691a2a-2a23-4999-8360-7fd815dfb2c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:00.197497 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6691a2a-2a23-4999-8360-7fd815dfb2c5 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:00.255002 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdfe6f7-4020-4a70-8c2a-bd4cb74611bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:00.256427 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdfe6f7-4020-4a70-8c2a-bd4cb74611bc service cinder] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:00.266919 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 904/1846] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:09:00 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:09:00.315566 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 943/1847] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:09:00 2022] POST /identity/v3/users => generated 289 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 03:09:00.317813 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdfe6f7-4020-4a70-8c2a-bd4cb74611bc service cinder] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:00.329872 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdfe6f7-4020-4a70-8c2a-bd4cb74611bc service cinder] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:00.334738 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f8491e-fd54-4b21-b662-a4e8abe319dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:00.335516 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f8491e-fd54-4b21-b662-a4e8abe319dd admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:00.335801 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f8491e-fd54-4b21-b662-a4e8abe319dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:00.336072 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f8491e-fd54-4b21-b662-a4e8abe319dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:00.336719 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f8491e-fd54-4b21-b662-a4e8abe319dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:00.338387 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f8491e-fd54-4b21-b662-a4e8abe319dd admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:00.348896 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 905/1848] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:09:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:09:00.353492 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6691a2a-2a23-4999-8360-7fd815dfb2c5 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:00.363907 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6691a2a-2a23-4999-8360-7fd815dfb2c5 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:00.369416 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5cc755-589e-4b58-bb37-077526395c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:00.370248 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5cc755-589e-4b58-bb37-077526395c83 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:00.370531 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5cc755-589e-4b58-bb37-077526395c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:00.375947 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5cc755-589e-4b58-bb37-077526395c83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:00.375947 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5cc755-589e-4b58-bb37-077526395c83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:00.375947 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5cc755-589e-4b58-bb37-077526395c83 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:00.383890 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 944/1849] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:09:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:09:00.388143 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f8491e-fd54-4b21-b662-a4e8abe319dd admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:00.398032 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f8491e-fd54-4b21-b662-a4e8abe319dd admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:00.402596 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741ef3ea-712a-4c7d-a63a-2c02303400a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:00.404780 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741ef3ea-712a-4c7d-a63a-2c02303400a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:00.404780 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741ef3ea-712a-4c7d-a63a-2c02303400a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:00.404780 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741ef3ea-712a-4c7d-a63a-2c02303400a8 admin admin] PATH_INFO: `/v3/projects/92db0aa26f764783b0e0ac950f0f0ec0/users/ca151c7629124671957d7366a584707c/roles/9bd0470b544240209a420f640c79c06c` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:00.418855 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741ef3ea-712a-4c7d-a63a-2c02303400a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92db0aa26f764783b0e0ac950f0f0ec0, user_id=ca151c7629124671957d7366a584707c, role_id=9bd0470b544240209a420f640c79c06c)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:00.422783 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741ef3ea-712a-4c7d-a63a-2c02303400a8 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:00.546739 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 906/1850] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:09:00 2022] PUT /identity/v3/projects/92db0aa26f764783b0e0ac950f0f0ec0/users/ca151c7629124671957d7366a584707c/roles/9bd0470b544240209a420f640c79c06c => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:09:00.557087 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316a7bee-b8fe-4b5d-a417-02cca66a14c3 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:00.557374 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316a7bee-b8fe-4b5d-a417-02cca66a14c3 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:00.557649 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316a7bee-b8fe-4b5d-a417-02cca66a14c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:00.574732 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:09:00.588059 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-316a7bee-b8fe-4b5d-a417-02cca66a14c3 None None] MFA Rules not processed for user `ca151c7629124671957d7366a584707c`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:09:00.589267 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-316a7bee-b8fe-4b5d-a417-02cca66a14c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:09:00.651223 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 945/1851] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:09:00 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:09:00.655766 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741ef3ea-712a-4c7d-a63a-2c02303400a8 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:00.720010 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741ef3ea-712a-4c7d-a63a-2c02303400a8 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:00.725077 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4ab78d-bbfb-4aff-896f-1295e54c794c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:00.726497 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4ab78d-bbfb-4aff-896f-1295e54c794c admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:00.726800 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4ab78d-bbfb-4aff-896f-1295e54c794c admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:00.727085 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4ab78d-bbfb-4aff-896f-1295e54c794c admin admin] PATH_INFO: `/v3/projects` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:00.727730 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4ab78d-bbfb-4aff-896f-1295e54c794c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:00.729733 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4ab78d-bbfb-4aff-896f-1295e54c794c admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:00.811690 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:09:00.827607 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 907/1852] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:09:00 2022] POST /identity/v3/projects => generated 377 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:09:00.831843 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316a7bee-b8fe-4b5d-a417-02cca66a14c3 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:00.845540 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316a7bee-b8fe-4b5d-a417-02cca66a14c3 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:00.871743 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4ab78d-bbfb-4aff-896f-1295e54c794c admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:00.873626 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-2f4ab78d-bbfb-4aff-896f-1295e54c794c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:09:00.919802 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435ca3c7-148b-4265-8d12-9f99e87327c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:00.920418 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435ca3c7-148b-4265-8d12-9f99e87327c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:00.920697 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435ca3c7-148b-4265-8d12-9f99e87327c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:00.920976 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435ca3c7-148b-4265-8d12-9f99e87327c5 admin admin] PATH_INFO: `/v3/users` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:00.921602 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435ca3c7-148b-4265-8d12-9f99e87327c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:00.931857 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435ca3c7-148b-4265-8d12-9f99e87327c5 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:00.935012 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4ab78d-bbfb-4aff-896f-1295e54c794c admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:00.939132 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acabb2ca-d865-4bd5-ace3-483f9a565ed1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab599d98b0624a2e89090312bcab713f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:00.939739 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acabb2ca-d865-4bd5-ace3-483f9a565ed1 service cinder] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:00.940026 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acabb2ca-d865-4bd5-ace3-483f9a565ed1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:00.940322 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acabb2ca-d865-4bd5-ace3-483f9a565ed1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:00.942134 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-acabb2ca-d865-4bd5-ace3-483f9a565ed1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:09:01.010258 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acabb2ca-d865-4bd5-ace3-483f9a565ed1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:01.011762 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acabb2ca-d865-4bd5-ace3-483f9a565ed1 service cinder] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:01.021253 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 908/1853] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:09:00 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 03:09:01.061474 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 946/1854] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:09:00 2022] POST /identity/v3/users => generated 289 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:09:01.067608 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acabb2ca-d865-4bd5-ace3-483f9a565ed1 service cinder] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:01.078600 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acabb2ca-d865-4bd5-ace3-483f9a565ed1 service cinder] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:01.083245 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68728ade-0135-449e-a7a8-8a3212d291d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:01.083845 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68728ade-0135-449e-a7a8-8a3212d291d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:01.084129 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68728ade-0135-449e-a7a8-8a3212d291d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:01.084400 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68728ade-0135-449e-a7a8-8a3212d291d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:01.085772 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68728ade-0135-449e-a7a8-8a3212d291d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:01.088073 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68728ade-0135-449e-a7a8-8a3212d291d0 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:01.097267 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 909/1855] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:09:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:09:01.101393 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435ca3c7-148b-4265-8d12-9f99e87327c5 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:01.111647 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435ca3c7-148b-4265-8d12-9f99e87327c5 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:01.116580 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0bb022-6c95-432e-919d-ded0e4c36e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:01.117197 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0bb022-6c95-432e-919d-ded0e4c36e37 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:01.117479 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0bb022-6c95-432e-919d-ded0e4c36e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:01.117770 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0bb022-6c95-432e-919d-ded0e4c36e37 admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:01.118437 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0bb022-6c95-432e-919d-ded0e4c36e37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:01.120070 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0bb022-6c95-432e-919d-ded0e4c36e37 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:01.130477 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 947/1856] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:09:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:09:01.135099 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68728ade-0135-449e-a7a8-8a3212d291d0 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:01.145030 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68728ade-0135-449e-a7a8-8a3212d291d0 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:01.150572 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689e83dd-391b-4ad8-9852-7c75be505a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:01.151225 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689e83dd-391b-4ad8-9852-7c75be505a57 admin admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:01.151522 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689e83dd-391b-4ad8-9852-7c75be505a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:01.151783 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689e83dd-391b-4ad8-9852-7c75be505a57 admin admin] PATH_INFO: `/v3/projects/9cedaf2f49fb4be4bed7d8139dbaebeb/users/c523b813985440af877e4b8e0600a533/roles/9bd0470b544240209a420f640c79c06c` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:01.166643 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689e83dd-391b-4ad8-9852-7c75be505a57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cedaf2f49fb4be4bed7d8139dbaebeb, user_id=c523b813985440af877e4b8e0600a533, role_id=9bd0470b544240209a420f640c79c06c)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:01.169309 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689e83dd-391b-4ad8-9852-7c75be505a57 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:01.272512 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 910/1857] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:09:01 2022] PUT /identity/v3/projects/9cedaf2f49fb4be4bed7d8139dbaebeb/users/c523b813985440af877e4b8e0600a533/roles/9bd0470b544240209a420f640c79c06c => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:09:01.283584 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f675f3-7b3e-46bd-b52d-62cece9d0c7f None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:01.283875 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f675f3-7b3e-46bd-b52d-62cece9d0c7f None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:01.284154 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f675f3-7b3e-46bd-b52d-62cece9d0c7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:01.299419 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:09:01.314405 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-c0f675f3-7b3e-46bd-b52d-62cece9d0c7f None None] MFA Rules not processed for user `c523b813985440af877e4b8e0600a533`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:09:01.315607 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-c0f675f3-7b3e-46bd-b52d-62cece9d0c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:09:01.381185 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 948/1858] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:09:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:09:01.386400 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689e83dd-391b-4ad8-9852-7c75be505a57 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:01.392787 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:09:01.457658 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689e83dd-391b-4ad8-9852-7c75be505a57 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:01.463147 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a03d89e-09be-411c-8332-ac65282412f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:01.463833 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03d89e-09be-411c-8332-ac65282412f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:01.464117 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03d89e-09be-411c-8332-ac65282412f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:01.464392 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03d89e-09be-411c-8332-ac65282412f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:01.465075 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a03d89e-09be-411c-8332-ac65282412f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:01.467593 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a03d89e-09be-411c-8332-ac65282412f2 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:01.594259 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 911/1859] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:09:01 2022] POST /identity/v3/projects => generated 377 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:09:01.598202 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f675f3-7b3e-46bd-b52d-62cece9d0c7f None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:01.610082 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f675f3-7b3e-46bd-b52d-62cece9d0c7f None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:01.669274 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49fd819-6c73-42d6-ac28-bd30047809d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:01.669928 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49fd819-6c73-42d6-ac28-bd30047809d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:01.671860 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49fd819-6c73-42d6-ac28-bd30047809d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:01.671860 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49fd819-6c73-42d6-ac28-bd30047809d7 admin admin] PATH_INFO: `/v3/users` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:01.671860 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49fd819-6c73-42d6-ac28-bd30047809d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:01.673014 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49fd819-6c73-42d6-ac28-bd30047809d7 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:01.780836 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 949/1860] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:09:01 2022] POST /identity/v3/users => generated 295 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:09:01.784870 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a03d89e-09be-411c-8332-ac65282412f2 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:01.795003 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a03d89e-09be-411c-8332-ac65282412f2 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:01.799582 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5255098c-dcc9-482c-9ae2-740d120cea86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:01.800175 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5255098c-dcc9-482c-9ae2-740d120cea86 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:01.800456 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5255098c-dcc9-482c-9ae2-740d120cea86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:01.800730 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5255098c-dcc9-482c-9ae2-740d120cea86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:01.801391 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5255098c-dcc9-482c-9ae2-740d120cea86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:01.803070 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5255098c-dcc9-482c-9ae2-740d120cea86 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:01.813611 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 912/1861] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:09:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:09:01.817570 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49fd819-6c73-42d6-ac28-bd30047809d7 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:01.828475 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49fd819-6c73-42d6-ac28-bd30047809d7 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:01.833084 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da14ed9-5031-491c-bcf1-606edc97ee46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:01.833779 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da14ed9-5031-491c-bcf1-606edc97ee46 admin admin] REQUEST_METHOD: `PUT` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:01.834065 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da14ed9-5031-491c-bcf1-606edc97ee46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:01.834340 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da14ed9-5031-491c-bcf1-606edc97ee46 admin admin] PATH_INFO: `/v3/projects/216b139cabcd48bda764e14e0fac68b2/users/aec225b6502141d59dfd6579db52ff4f/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:01.851833 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da14ed9-5031-491c-bcf1-606edc97ee46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=216b139cabcd48bda764e14e0fac68b2, user_id=aec225b6502141d59dfd6579db52ff4f, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:01.854650 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da14ed9-5031-491c-bcf1-606edc97ee46 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:01.941590 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 950/1862] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:09:01 2022] PUT /identity/v3/projects/216b139cabcd48bda764e14e0fac68b2/users/aec225b6502141d59dfd6579db52ff4f/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:09:01.948918 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5255098c-dcc9-482c-9ae2-740d120cea86 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:01.959241 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5255098c-dcc9-482c-9ae2-740d120cea86 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:02.018017 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a84e669-9e25-46fa-a567-5fc38701b66e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:02.018663 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a84e669-9e25-46fa-a567-5fc38701b66e admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:02.018942 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a84e669-9e25-46fa-a567-5fc38701b66e admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:02.019384 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a84e669-9e25-46fa-a567-5fc38701b66e admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:02.020035 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a84e669-9e25-46fa-a567-5fc38701b66e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:02.021663 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a84e669-9e25-46fa-a567-5fc38701b66e admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:02.031680 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 913/1863] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:09:01 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:09:02.038056 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da14ed9-5031-491c-bcf1-606edc97ee46 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:02.049261 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da14ed9-5031-491c-bcf1-606edc97ee46 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:02.057535 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f34765-8ae2-4010-88d4-b329cd1de88c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:02.058331 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f34765-8ae2-4010-88d4-b329cd1de88c admin admin] REQUEST_METHOD: `PUT` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:02.058613 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f34765-8ae2-4010-88d4-b329cd1de88c admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:02.058888 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f34765-8ae2-4010-88d4-b329cd1de88c admin admin] PATH_INFO: `/v3/projects/216b139cabcd48bda764e14e0fac68b2/users/aec225b6502141d59dfd6579db52ff4f/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:02.061723 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f34765-8ae2-4010-88d4-b329cd1de88c admin admin] RBAC: Authorizing `identity:create_grant(project_id=216b139cabcd48bda764e14e0fac68b2, user_id=aec225b6502141d59dfd6579db52ff4f, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:02.065312 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f34765-8ae2-4010-88d4-b329cd1de88c admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:02.078160 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 951/1864] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:09:02 2022] PUT /identity/v3/projects/216b139cabcd48bda764e14e0fac68b2/users/aec225b6502141d59dfd6579db52ff4f/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:09:02.148243 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7204ba4c-9058-42b6-94b8-d1ed8fe09ecb None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:02.148597 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7204ba4c-9058-42b6-94b8-d1ed8fe09ecb None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:02.148887 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7204ba4c-9058-42b6-94b8-d1ed8fe09ecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:02.178265 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-7204ba4c-9058-42b6-94b8-d1ed8fe09ecb None None] MFA Rules not processed for user `aec225b6502141d59dfd6579db52ff4f`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:09:02.180012 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-7204ba4c-9058-42b6-94b8-d1ed8fe09ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:09:02.250565 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:09:02.254608 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 914/1865] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:09:02 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 03:09:02.266970 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f34765-8ae2-4010-88d4-b329cd1de88c admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:02.323269 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:09:02.327082 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f34765-8ae2-4010-88d4-b329cd1de88c admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:02.330746 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd726dc-204c-4334-8d75-dd1a03333c69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab599d98b0624a2e89090312bcab713f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:02.331345 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd726dc-204c-4334-8d75-dd1a03333c69 service cinder] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:02.331620 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd726dc-204c-4334-8d75-dd1a03333c69 service cinder] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:02.331890 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd726dc-204c-4334-8d75-dd1a03333c69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:02.389501 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd726dc-204c-4334-8d75-dd1a03333c69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:02.390975 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd726dc-204c-4334-8d75-dd1a03333c69 service cinder] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:02.399671 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 952/1866] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:09:02 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:09:04.546417 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7204ba4c-9058-42b6-94b8-d1ed8fe09ecb None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:04.555794 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7204ba4c-9058-42b6-94b8-d1ed8fe09ecb None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:04.559417 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320059d0-9e57-40f9-b3f6-76df14ad04b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab599d98b0624a2e89090312bcab713f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:04.560007 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320059d0-9e57-40f9-b3f6-76df14ad04b5 service cinder] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:04.561888 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320059d0-9e57-40f9-b3f6-76df14ad04b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:04.561888 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320059d0-9e57-40f9-b3f6-76df14ad04b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:04.617830 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320059d0-9e57-40f9-b3f6-76df14ad04b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:04.619399 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320059d0-9e57-40f9-b3f6-76df14ad04b5 service cinder] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:04.635919 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 915/1867] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:09:04 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:09:06.248930 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd726dc-204c-4334-8d75-dd1a03333c69 service cinder] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:06.258548 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd726dc-204c-4334-8d75-dd1a03333c69 service cinder] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:06.262640 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaee07ec-57dc-4519-965a-232474fbd1ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab599d98b0624a2e89090312bcab713f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:06.264052 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaee07ec-57dc-4519-965a-232474fbd1ae service cinder] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:06.264052 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaee07ec-57dc-4519-965a-232474fbd1ae service cinder] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:06.264052 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaee07ec-57dc-4519-965a-232474fbd1ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:06.275236 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaee07ec-57dc-4519-965a-232474fbd1ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:06.276642 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaee07ec-57dc-4519-965a-232474fbd1ae service cinder] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:06.294396 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 953/1868] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:09:06 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:09:14.626029 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=28 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:09:14.700466 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320059d0-9e57-40f9-b3f6-76df14ad04b5 service cinder] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:14.766725 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320059d0-9e57-40f9-b3f6-76df14ad04b5 service cinder] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:14.771972 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46f85bc-0aa0-466b-90b1-7a09acf0aec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:14.772619 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46f85bc-0aa0-466b-90b1-7a09acf0aec7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:14.772901 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46f85bc-0aa0-466b-90b1-7a09acf0aec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:14.773180 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46f85bc-0aa0-466b-90b1-7a09acf0aec7 admin admin] PATH_INFO: `/v3/users/ca151c7629124671957d7366a584707c` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:14.775203 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46f85bc-0aa0-466b-90b1-7a09acf0aec7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca151c7629124671957d7366a584707c)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:14.778186 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46f85bc-0aa0-466b-90b1-7a09acf0aec7 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:15.021779 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-b46f85bc-0aa0-466b-90b1-7a09acf0aec7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca151c7629124671957d7366a584707c'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:15.031015 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-b46f85bc-0aa0-466b-90b1-7a09acf0aec7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca151c7629124671957d7366a584707c'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:15.044053 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-b46f85bc-0aa0-466b-90b1-7a09acf0aec7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca151c7629124671957d7366a584707c'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:15.045169 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-b46f85bc-0aa0-466b-90b1-7a09acf0aec7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca151c7629124671957d7366a584707c'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:15.123930 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-b46f85bc-0aa0-466b-90b1-7a09acf0aec7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca151c7629124671957d7366a584707c'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:15.130649 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 916/1869] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:09:14 2022] DELETE /identity/v3/users/ca151c7629124671957d7366a584707c => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:09:15.135669 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecbcef6-5742-4d67-a3e7-6d1b4a6ea510 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:15.136133 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecbcef6-5742-4d67-a3e7-6d1b4a6ea510 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:15.136224 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecbcef6-5742-4d67-a3e7-6d1b4a6ea510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:15.155842 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-eecbcef6-5742-4d67-a3e7-6d1b4a6ea510 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:09:15.157057 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-eecbcef6-5742-4d67-a3e7-6d1b4a6ea510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:09:15.216647 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 954/1870] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:09:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:09:15.232978 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46f85bc-0aa0-466b-90b1-7a09acf0aec7 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:15.232978 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-b46f85bc-0aa0-466b-90b1-7a09acf0aec7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:09:15.289336 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46f85bc-0aa0-466b-90b1-7a09acf0aec7 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:15.293085 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83557325-a2db-4b7f-b908-e5ccea940d40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:15.293695 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83557325-a2db-4b7f-b908-e5ccea940d40 service neutron] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:15.294193 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83557325-a2db-4b7f-b908-e5ccea940d40 service neutron] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:15.294477 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83557325-a2db-4b7f-b908-e5ccea940d40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:15.308238 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83557325-a2db-4b7f-b908-e5ccea940d40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:15.309707 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83557325-a2db-4b7f-b908-e5ccea940d40 service neutron] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:15.319600 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 917/1871] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:09:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:09:15.820295 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecbcef6-5742-4d67-a3e7-6d1b4a6ea510 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:15.821727 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-eecbcef6-5742-4d67-a3e7-6d1b4a6ea510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:09:15.842645 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecbcef6-5742-4d67-a3e7-6d1b4a6ea510 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:15.849041 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08471d38-a141-4c1f-813d-9827fd90fcea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:15.849597 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08471d38-a141-4c1f-813d-9827fd90fcea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:15.849916 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08471d38-a141-4c1f-813d-9827fd90fcea admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:15.850192 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08471d38-a141-4c1f-813d-9827fd90fcea admin admin] PATH_INFO: `/v3/projects/92db0aa26f764783b0e0ac950f0f0ec0` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:15.851364 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08471d38-a141-4c1f-813d-9827fd90fcea admin admin] RBAC: Authorizing `identity:delete_project(project_id=92db0aa26f764783b0e0ac950f0f0ec0)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:15.854443 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08471d38-a141-4c1f-813d-9827fd90fcea admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:15.968835 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-08471d38-a141-4c1f-813d-9827fd90fcea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92db0aa26f764783b0e0ac950f0f0ec0', 'request_id': 'req-08471d38-a141-4c1f-813d-9827fd90fcea'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:15.981450 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-08471d38-a141-4c1f-813d-9827fd90fcea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92db0aa26f764783b0e0ac950f0f0ec0', 'request_id': 'req-08471d38-a141-4c1f-813d-9827fd90fcea'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:16.048465 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-08471d38-a141-4c1f-813d-9827fd90fcea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92db0aa26f764783b0e0ac950f0f0ec0', 'request_id': 'req-08471d38-a141-4c1f-813d-9827fd90fcea'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:16.062180 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-08471d38-a141-4c1f-813d-9827fd90fcea admin admin] The token cache is being invalidate because project 92db0aa26f764783b0e0ac950f0f0ec0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81517) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:09:16.062488 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-08471d38-a141-4c1f-813d-9827fd90fcea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:16.063129 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-08471d38-a141-4c1f-813d-9827fd90fcea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:16.064779 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 955/1872] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:09:15 2022] DELETE /identity/v3/projects/92db0aa26f764783b0e0ac950f0f0ec0 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:09:16.068779 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83557325-a2db-4b7f-b908-e5ccea940d40 service neutron] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:16.078475 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83557325-a2db-4b7f-b908-e5ccea940d40 service neutron] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:16.097267 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:09:16.140061 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca5a8e3-99f1-4cdd-8c3d-c2f34c103753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:16.140680 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca5a8e3-99f1-4cdd-8c3d-c2f34c103753 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:16.140962 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca5a8e3-99f1-4cdd-8c3d-c2f34c103753 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:16.141238 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca5a8e3-99f1-4cdd-8c3d-c2f34c103753 admin admin] PATH_INFO: `/v3/users/c523b813985440af877e4b8e0600a533` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:16.142356 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca5a8e3-99f1-4cdd-8c3d-c2f34c103753 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c523b813985440af877e4b8e0600a533)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:16.144279 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca5a8e3-99f1-4cdd-8c3d-c2f34c103753 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:16.383128 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-dca5a8e3-99f1-4cdd-8c3d-c2f34c103753 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c523b813985440af877e4b8e0600a533'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:16.390024 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-dca5a8e3-99f1-4cdd-8c3d-c2f34c103753 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c523b813985440af877e4b8e0600a533'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:16.402494 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-dca5a8e3-99f1-4cdd-8c3d-c2f34c103753 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c523b813985440af877e4b8e0600a533'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:16.403192 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-dca5a8e3-99f1-4cdd-8c3d-c2f34c103753 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c523b813985440af877e4b8e0600a533'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:16.482978 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-dca5a8e3-99f1-4cdd-8c3d-c2f34c103753 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c523b813985440af877e4b8e0600a533'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:16.492907 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 918/1873] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:09:16 2022] DELETE /identity/v3/users/c523b813985440af877e4b8e0600a533 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:09:16.925376 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08471d38-a141-4c1f-813d-9827fd90fcea admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:16.927393 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-08471d38-a141-4c1f-813d-9827fd90fcea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:09:16.946725 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08471d38-a141-4c1f-813d-9827fd90fcea admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:16.956615 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:09:17.005038 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d404b1e3-0603-4084-acb4-b4ebc582ef35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:17.005690 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d404b1e3-0603-4084-acb4-b4ebc582ef35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:17.005996 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d404b1e3-0603-4084-acb4-b4ebc582ef35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:17.006315 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d404b1e3-0603-4084-acb4-b4ebc582ef35 admin admin] PATH_INFO: `/v3/projects/9cedaf2f49fb4be4bed7d8139dbaebeb` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:17.007456 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d404b1e3-0603-4084-acb4-b4ebc582ef35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9cedaf2f49fb4be4bed7d8139dbaebeb)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:17.014565 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d404b1e3-0603-4084-acb4-b4ebc582ef35 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:17.169397 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-d404b1e3-0603-4084-acb4-b4ebc582ef35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9cedaf2f49fb4be4bed7d8139dbaebeb', 'request_id': 'req-d404b1e3-0603-4084-acb4-b4ebc582ef35'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:17.175707 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-d404b1e3-0603-4084-acb4-b4ebc582ef35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9cedaf2f49fb4be4bed7d8139dbaebeb', 'request_id': 'req-d404b1e3-0603-4084-acb4-b4ebc582ef35'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:17.237593 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-d404b1e3-0603-4084-acb4-b4ebc582ef35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9cedaf2f49fb4be4bed7d8139dbaebeb', 'request_id': 'req-d404b1e3-0603-4084-acb4-b4ebc582ef35'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:17.248890 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-d404b1e3-0603-4084-acb4-b4ebc582ef35 admin admin] The token cache is being invalidate because project 9cedaf2f49fb4be4bed7d8139dbaebeb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81517) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:09:17.249192 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-d404b1e3-0603-4084-acb4-b4ebc582ef35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:17.249884 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-d404b1e3-0603-4084-acb4-b4ebc582ef35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:17.251520 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 956/1874] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:09:16 2022] DELETE /identity/v3/projects/9cedaf2f49fb4be4bed7d8139dbaebeb => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:09:17.255630 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca5a8e3-99f1-4cdd-8c3d-c2f34c103753 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:17.257033 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-dca5a8e3-99f1-4cdd-8c3d-c2f34c103753 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:09:17.269542 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca5a8e3-99f1-4cdd-8c3d-c2f34c103753 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:17.324607 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10dc7fd-0ccb-432f-9b0d-45b20c3ed9b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:17.325236 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10dc7fd-0ccb-432f-9b0d-45b20c3ed9b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:17.325518 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10dc7fd-0ccb-432f-9b0d-45b20c3ed9b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:17.325818 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10dc7fd-0ccb-432f-9b0d-45b20c3ed9b8 admin admin] PATH_INFO: `/v3/users/aec225b6502141d59dfd6579db52ff4f` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:17.326930 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10dc7fd-0ccb-432f-9b0d-45b20c3ed9b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aec225b6502141d59dfd6579db52ff4f)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:17.328849 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10dc7fd-0ccb-432f-9b0d-45b20c3ed9b8 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:17.495207 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:09:17.497639 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-b10dc7fd-0ccb-432f-9b0d-45b20c3ed9b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aec225b6502141d59dfd6579db52ff4f'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:17.504755 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-b10dc7fd-0ccb-432f-9b0d-45b20c3ed9b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aec225b6502141d59dfd6579db52ff4f'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:17.516478 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-b10dc7fd-0ccb-432f-9b0d-45b20c3ed9b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aec225b6502141d59dfd6579db52ff4f'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:17.517168 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-b10dc7fd-0ccb-432f-9b0d-45b20c3ed9b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aec225b6502141d59dfd6579db52ff4f'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:17.620971 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-b10dc7fd-0ccb-432f-9b0d-45b20c3ed9b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aec225b6502141d59dfd6579db52ff4f'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:17.627596 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 919/1875] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:09:17 2022] DELETE /identity/v3/users/aec225b6502141d59dfd6579db52ff4f => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:09:18.120953 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d404b1e3-0603-4084-acb4-b4ebc582ef35 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:18.122439 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-d404b1e3-0603-4084-acb4-b4ebc582ef35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:09:18.135830 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d404b1e3-0603-4084-acb4-b4ebc582ef35 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:18.196447 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af395dfb-ea2b-456f-bbc3-eab1bae0a92c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:18.197080 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af395dfb-ea2b-456f-bbc3-eab1bae0a92c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:18.197361 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af395dfb-ea2b-456f-bbc3-eab1bae0a92c admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:18.197638 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af395dfb-ea2b-456f-bbc3-eab1bae0a92c admin admin] PATH_INFO: `/v3/projects/216b139cabcd48bda764e14e0fac68b2` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:18.198806 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af395dfb-ea2b-456f-bbc3-eab1bae0a92c admin admin] RBAC: Authorizing `identity:delete_project(project_id=216b139cabcd48bda764e14e0fac68b2)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:18.200718 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af395dfb-ea2b-456f-bbc3-eab1bae0a92c admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:18.334812 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:09:18.337750 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-af395dfb-ea2b-456f-bbc3-eab1bae0a92c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '216b139cabcd48bda764e14e0fac68b2', 'request_id': 'req-af395dfb-ea2b-456f-bbc3-eab1bae0a92c'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:18.344593 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-af395dfb-ea2b-456f-bbc3-eab1bae0a92c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '216b139cabcd48bda764e14e0fac68b2', 'request_id': 'req-af395dfb-ea2b-456f-bbc3-eab1bae0a92c'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:18.426208 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-af395dfb-ea2b-456f-bbc3-eab1bae0a92c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '216b139cabcd48bda764e14e0fac68b2', 'request_id': 'req-af395dfb-ea2b-456f-bbc3-eab1bae0a92c'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:18.441001 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-af395dfb-ea2b-456f-bbc3-eab1bae0a92c admin admin] The token cache is being invalidate because project 216b139cabcd48bda764e14e0fac68b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81517) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:09:18.441296 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-af395dfb-ea2b-456f-bbc3-eab1bae0a92c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:18.442005 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-af395dfb-ea2b-456f-bbc3-eab1bae0a92c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:18.444013 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 957/1876] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:09:18 2022] DELETE /identity/v3/projects/216b139cabcd48bda764e14e0fac68b2 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:09:24.589826 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10dc7fd-0ccb-432f-9b0d-45b20c3ed9b8 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:24.591380 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-b10dc7fd-0ccb-432f-9b0d-45b20c3ed9b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:09:24.604260 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10dc7fd-0ccb-432f-9b0d-45b20c3ed9b8 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:24.662593 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d173fdc4-4df5-41f5-a78c-c91cbc8fb7ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:24.663228 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d173fdc4-4df5-41f5-a78c-c91cbc8fb7ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:24.663507 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d173fdc4-4df5-41f5-a78c-c91cbc8fb7ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:24.663782 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d173fdc4-4df5-41f5-a78c-c91cbc8fb7ae admin admin] PATH_INFO: `/v3/users/f3d01587bbde467f8e26dfb399bd4a32` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:24.664892 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d173fdc4-4df5-41f5-a78c-c91cbc8fb7ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3d01587bbde467f8e26dfb399bd4a32)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:24.666868 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d173fdc4-4df5-41f5-a78c-c91cbc8fb7ae admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:24.844622 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-d173fdc4-4df5-41f5-a78c-c91cbc8fb7ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3d01587bbde467f8e26dfb399bd4a32'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:24.851677 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-d173fdc4-4df5-41f5-a78c-c91cbc8fb7ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3d01587bbde467f8e26dfb399bd4a32'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:24.865713 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-d173fdc4-4df5-41f5-a78c-c91cbc8fb7ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3d01587bbde467f8e26dfb399bd4a32'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:24.866431 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-d173fdc4-4df5-41f5-a78c-c91cbc8fb7ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3d01587bbde467f8e26dfb399bd4a32'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:24.921969 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-d173fdc4-4df5-41f5-a78c-c91cbc8fb7ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3d01587bbde467f8e26dfb399bd4a32'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:24.929970 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 920/1877] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:09:24 2022] DELETE /identity/v3/users/f3d01587bbde467f8e26dfb399bd4a32 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:09:24.935321 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e21b3aa-a74b-4df7-9176-96fe05aa7004 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:24.935872 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e21b3aa-a74b-4df7-9176-96fe05aa7004 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:24.935973 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e21b3aa-a74b-4df7-9176-96fe05aa7004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:24.956278 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-5e21b3aa-a74b-4df7-9176-96fe05aa7004 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:09:24.957398 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-5e21b3aa-a74b-4df7-9176-96fe05aa7004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:09:25.028140 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 958/1878] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:09:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:09:25.037733 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d173fdc4-4df5-41f5-a78c-c91cbc8fb7ae admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:25.039158 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-d173fdc4-4df5-41f5-a78c-c91cbc8fb7ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:09:25.087297 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:09:25.101189 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d173fdc4-4df5-41f5-a78c-c91cbc8fb7ae admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:25.104801 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb93796-362c-4636-a9da-306aa56635ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:25.106857 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb93796-362c-4636-a9da-306aa56635ef service neutron] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:25.106857 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb93796-362c-4636-a9da-306aa56635ef service neutron] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:25.106857 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb93796-362c-4636-a9da-306aa56635ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:25.115738 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb93796-362c-4636-a9da-306aa56635ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:25.117157 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb93796-362c-4636-a9da-306aa56635ef service neutron] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:25.126580 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 921/1879] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:09:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:09:25.537071 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e21b3aa-a74b-4df7-9176-96fe05aa7004 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:25.538606 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-5e21b3aa-a74b-4df7-9176-96fe05aa7004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:09:25.551382 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e21b3aa-a74b-4df7-9176-96fe05aa7004 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:25.556777 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c71e52-8b00-40ed-a9bb-dd10a8d6dfd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:25.557584 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c71e52-8b00-40ed-a9bb-dd10a8d6dfd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:25.557727 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c71e52-8b00-40ed-a9bb-dd10a8d6dfd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:25.558055 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c71e52-8b00-40ed-a9bb-dd10a8d6dfd4 admin admin] PATH_INFO: `/v3/projects/087d77718a3d414dace397802d70873f` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:25.560436 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c71e52-8b00-40ed-a9bb-dd10a8d6dfd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=087d77718a3d414dace397802d70873f)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:25.561381 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c71e52-8b00-40ed-a9bb-dd10a8d6dfd4 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:25.677410 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-23c71e52-8b00-40ed-a9bb-dd10a8d6dfd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '087d77718a3d414dace397802d70873f', 'request_id': 'req-23c71e52-8b00-40ed-a9bb-dd10a8d6dfd4'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:25.683741 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-23c71e52-8b00-40ed-a9bb-dd10a8d6dfd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '087d77718a3d414dace397802d70873f', 'request_id': 'req-23c71e52-8b00-40ed-a9bb-dd10a8d6dfd4'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:25.751616 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-23c71e52-8b00-40ed-a9bb-dd10a8d6dfd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '087d77718a3d414dace397802d70873f', 'request_id': 'req-23c71e52-8b00-40ed-a9bb-dd10a8d6dfd4'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:25.764033 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-23c71e52-8b00-40ed-a9bb-dd10a8d6dfd4 admin admin] The token cache is being invalidate because project 087d77718a3d414dace397802d70873f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81517) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:09:25.764396 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-23c71e52-8b00-40ed-a9bb-dd10a8d6dfd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:25.764998 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-23c71e52-8b00-40ed-a9bb-dd10a8d6dfd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:25.766543 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 959/1880] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:09:25 2022] DELETE /identity/v3/projects/087d77718a3d414dace397802d70873f => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:09:25.770564 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb93796-362c-4636-a9da-306aa56635ef service neutron] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:25.780859 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb93796-362c-4636-a9da-306aa56635ef service neutron] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:25.837294 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d90913-b16a-44ee-b753-762d1eb397ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:25.837964 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d90913-b16a-44ee-b753-762d1eb397ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:25.838245 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d90913-b16a-44ee-b753-762d1eb397ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:25.838518 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d90913-b16a-44ee-b753-762d1eb397ec admin admin] PATH_INFO: `/v3/users/8b3e86586030436c97bb45f1cb056348` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:25.839632 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d90913-b16a-44ee-b753-762d1eb397ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b3e86586030436c97bb45f1cb056348)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:25.841574 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d90913-b16a-44ee-b753-762d1eb397ec admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:25.988573 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-84d90913-b16a-44ee-b753-762d1eb397ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b3e86586030436c97bb45f1cb056348'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:25.995415 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-84d90913-b16a-44ee-b753-762d1eb397ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b3e86586030436c97bb45f1cb056348'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:26.015326 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-84d90913-b16a-44ee-b753-762d1eb397ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b3e86586030436c97bb45f1cb056348'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:26.016630 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-84d90913-b16a-44ee-b753-762d1eb397ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b3e86586030436c97bb45f1cb056348'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:26.083630 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-84d90913-b16a-44ee-b753-762d1eb397ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b3e86586030436c97bb45f1cb056348'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:26.090231 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 922/1881] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:09:25 2022] DELETE /identity/v3/users/8b3e86586030436c97bb45f1cb056348 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:09:26.519346 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c71e52-8b00-40ed-a9bb-dd10a8d6dfd4 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:26.520839 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-23c71e52-8b00-40ed-a9bb-dd10a8d6dfd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:09:26.535233 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c71e52-8b00-40ed-a9bb-dd10a8d6dfd4 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:26.588396 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:09:26.594741 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69dabe4-2555-4ea0-a824-073cf0e8dc17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:26.595399 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69dabe4-2555-4ea0-a824-073cf0e8dc17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:26.595680 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69dabe4-2555-4ea0-a824-073cf0e8dc17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:26.595955 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69dabe4-2555-4ea0-a824-073cf0e8dc17 admin admin] PATH_INFO: `/v3/projects/36f2b1a55312424d9246dc70f6517e84` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:26.597104 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69dabe4-2555-4ea0-a824-073cf0e8dc17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=36f2b1a55312424d9246dc70f6517e84)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:26.599041 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69dabe4-2555-4ea0-a824-073cf0e8dc17 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:26.706303 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-d69dabe4-2555-4ea0-a824-073cf0e8dc17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36f2b1a55312424d9246dc70f6517e84', 'request_id': 'req-d69dabe4-2555-4ea0-a824-073cf0e8dc17'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:26.712718 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-d69dabe4-2555-4ea0-a824-073cf0e8dc17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36f2b1a55312424d9246dc70f6517e84', 'request_id': 'req-d69dabe4-2555-4ea0-a824-073cf0e8dc17'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:26.768674 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-d69dabe4-2555-4ea0-a824-073cf0e8dc17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36f2b1a55312424d9246dc70f6517e84', 'request_id': 'req-d69dabe4-2555-4ea0-a824-073cf0e8dc17'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:26.780307 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-d69dabe4-2555-4ea0-a824-073cf0e8dc17 admin admin] The token cache is being invalidate because project 36f2b1a55312424d9246dc70f6517e84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81517) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:09:26.780727 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-d69dabe4-2555-4ea0-a824-073cf0e8dc17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:26.781381 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-d69dabe4-2555-4ea0-a824-073cf0e8dc17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:26.783015 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 960/1882] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:09:26 2022] DELETE /identity/v3/projects/36f2b1a55312424d9246dc70f6517e84 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:09:26.788179 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddf9c8b-ab7e-4560-9f82-d8033000dead None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:26.788670 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddf9c8b-ab7e-4560-9f82-d8033000dead None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:26.788780 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddf9c8b-ab7e-4560-9f82-d8033000dead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:26.809341 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-5ddf9c8b-ab7e-4560-9f82-d8033000dead None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:09:26.810639 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-5ddf9c8b-ab7e-4560-9f82-d8033000dead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:09:26.869027 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 923/1883] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:09:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:09:26.876641 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efa4fb1-bf37-4374-a661-1ad1218bb6d8 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:26.876934 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efa4fb1-bf37-4374-a661-1ad1218bb6d8 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:26.877209 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efa4fb1-bf37-4374-a661-1ad1218bb6d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:26.897595 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-5efa4fb1-bf37-4374-a661-1ad1218bb6d8 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:09:26.898789 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-5efa4fb1-bf37-4374-a661-1ad1218bb6d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:09:26.907580 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 961/1884] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:09:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:09:26.911814 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddf9c8b-ab7e-4560-9f82-d8033000dead None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:26.916586 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:09:26.921878 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddf9c8b-ab7e-4560-9f82-d8033000dead None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:26.926461 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7437399-ab09-4e72-a235-49a01011ccfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:26.939775 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7437399-ab09-4e72-a235-49a01011ccfa admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:26.939775 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7437399-ab09-4e72-a235-49a01011ccfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:26.939775 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7437399-ab09-4e72-a235-49a01011ccfa admin admin] PATH_INFO: `/v3/domains` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:26.939775 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7437399-ab09-4e72-a235-49a01011ccfa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 03:09:26.939775 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7437399-ab09-4e72-a235-49a01011ccfa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:26.939775 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7437399-ab09-4e72-a235-49a01011ccfa admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:26.941331 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 924/1885] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Oct 14 03:09:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 03:09:26.945711 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efa4fb1-bf37-4374-a661-1ad1218bb6d8 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:26.955547 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efa4fb1-bf37-4374-a661-1ad1218bb6d8 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:26.960081 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7acb13-443c-46de-86a6-3d9e88af809a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:26.964599 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7acb13-443c-46de-86a6-3d9e88af809a admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:26.964599 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7acb13-443c-46de-86a6-3d9e88af809a admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:26.964599 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7acb13-443c-46de-86a6-3d9e88af809a admin admin] PATH_INFO: `/v3/projects` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:26.964599 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7acb13-443c-46de-86a6-3d9e88af809a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:26.964599 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7acb13-443c-46de-86a6-3d9e88af809a admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:27.059363 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 962/1886] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:09:26 2022] POST /identity/v3/projects => generated 381 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:09:27.065260 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7437399-ab09-4e72-a235-49a01011ccfa admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:27.075491 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7437399-ab09-4e72-a235-49a01011ccfa admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:27.149551 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a779c5-fdc7-43ec-be24-f404ad1e7ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:27.150136 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a779c5-fdc7-43ec-be24-f404ad1e7ef8 admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:27.150418 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a779c5-fdc7-43ec-be24-f404ad1e7ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:27.150691 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a779c5-fdc7-43ec-be24-f404ad1e7ef8 admin admin] PATH_INFO: `/v3/users` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:27.151455 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a779c5-fdc7-43ec-be24-f404ad1e7ef8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:27.153407 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a779c5-fdc7-43ec-be24-f404ad1e7ef8 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:27.224637 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 925/1887] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:09:27 2022] POST /identity/v3/users => generated 291 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:09:27.228571 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7acb13-443c-46de-86a6-3d9e88af809a admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:27.239886 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7acb13-443c-46de-86a6-3d9e88af809a admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:27.244577 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afda58b-a4e8-487c-aa23-c0bc0847f12e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:27.245168 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afda58b-a4e8-487c-aa23-c0bc0847f12e admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:27.245449 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afda58b-a4e8-487c-aa23-c0bc0847f12e admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:27.245722 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afda58b-a4e8-487c-aa23-c0bc0847f12e admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:27.247046 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afda58b-a4e8-487c-aa23-c0bc0847f12e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:27.248692 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afda58b-a4e8-487c-aa23-c0bc0847f12e admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:27.258621 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 963/1888] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:09:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:09:27.262406 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a779c5-fdc7-43ec-be24-f404ad1e7ef8 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:27.273697 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a779c5-fdc7-43ec-be24-f404ad1e7ef8 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:27.279905 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7268e7fe-649c-4c2f-bc72-94c11b8d3778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:27.280679 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7268e7fe-649c-4c2f-bc72-94c11b8d3778 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:27.280964 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7268e7fe-649c-4c2f-bc72-94c11b8d3778 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:27.281240 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7268e7fe-649c-4c2f-bc72-94c11b8d3778 admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:27.281931 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7268e7fe-649c-4c2f-bc72-94c11b8d3778 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:27.283545 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7268e7fe-649c-4c2f-bc72-94c11b8d3778 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:27.296516 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 926/1889] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:09:27 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:09:27.300629 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afda58b-a4e8-487c-aa23-c0bc0847f12e admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:27.310671 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afda58b-a4e8-487c-aa23-c0bc0847f12e admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:27.315467 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea745779-7d3b-4686-8b61-bcb86298546c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:27.316113 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea745779-7d3b-4686-8b61-bcb86298546c admin admin] REQUEST_METHOD: `PUT` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:27.316398 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea745779-7d3b-4686-8b61-bcb86298546c admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:27.316667 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea745779-7d3b-4686-8b61-bcb86298546c admin admin] PATH_INFO: `/v3/projects/fb434417cb0a427eb1fdbfde98c5de51/users/8fea54e099f449c1887e34af8668ebfa/roles/9bd0470b544240209a420f640c79c06c` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:27.331087 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea745779-7d3b-4686-8b61-bcb86298546c admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb434417cb0a427eb1fdbfde98c5de51, user_id=8fea54e099f449c1887e34af8668ebfa, role_id=9bd0470b544240209a420f640c79c06c)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:27.333742 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea745779-7d3b-4686-8b61-bcb86298546c admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:27.409318 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 964/1890] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:09:27 2022] PUT /identity/v3/projects/fb434417cb0a427eb1fdbfde98c5de51/users/8fea54e099f449c1887e34af8668ebfa/roles/9bd0470b544240209a420f640c79c06c => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:09:27.421279 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9064bd74-2071-48ce-acbf-fba7478dfd3a None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:27.421562 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9064bd74-2071-48ce-acbf-fba7478dfd3a None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:27.421876 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9064bd74-2071-48ce-acbf-fba7478dfd3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:27.452796 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-9064bd74-2071-48ce-acbf-fba7478dfd3a None None] MFA Rules not processed for user `8fea54e099f449c1887e34af8668ebfa`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:09:27.454045 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-9064bd74-2071-48ce-acbf-fba7478dfd3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:09:27.518923 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 927/1891] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:09:27 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:09:27.523384 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea745779-7d3b-4686-8b61-bcb86298546c admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:27.584227 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea745779-7d3b-4686-8b61-bcb86298546c admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:27.588934 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67cc757-b2c9-4606-8e9b-2de112280161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:27.589530 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67cc757-b2c9-4606-8e9b-2de112280161 admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:27.590342 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67cc757-b2c9-4606-8e9b-2de112280161 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:27.590619 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67cc757-b2c9-4606-8e9b-2de112280161 admin admin] PATH_INFO: `/v3/projects` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:27.591256 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67cc757-b2c9-4606-8e9b-2de112280161 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:27.593153 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67cc757-b2c9-4606-8e9b-2de112280161 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:27.656930 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 965/1892] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:09:27 2022] POST /identity/v3/projects => generated 381 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:09:27.664502 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9064bd74-2071-48ce-acbf-fba7478dfd3a None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:27.675009 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9064bd74-2071-48ce-acbf-fba7478dfd3a None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:27.724142 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:09:27.729774 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c40edb-47c8-45b3-bd82-4dff43d5c810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:27.730382 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c40edb-47c8-45b3-bd82-4dff43d5c810 admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:27.730679 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c40edb-47c8-45b3-bd82-4dff43d5c810 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:27.730951 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c40edb-47c8-45b3-bd82-4dff43d5c810 admin admin] PATH_INFO: `/v3/users` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:27.731571 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c40edb-47c8-45b3-bd82-4dff43d5c810 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:27.733468 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c40edb-47c8-45b3-bd82-4dff43d5c810 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:27.791880 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 928/1893] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:09:27 2022] POST /identity/v3/users => generated 297 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:09:27.795952 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67cc757-b2c9-4606-8e9b-2de112280161 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:27.806789 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67cc757-b2c9-4606-8e9b-2de112280161 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:27.811664 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8249ca32-8186-4723-b847-591c5f83c574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:27.812289 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8249ca32-8186-4723-b847-591c5f83c574 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:27.812571 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8249ca32-8186-4723-b847-591c5f83c574 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:27.812846 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8249ca32-8186-4723-b847-591c5f83c574 admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:27.813483 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8249ca32-8186-4723-b847-591c5f83c574 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:27.815135 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8249ca32-8186-4723-b847-591c5f83c574 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:27.817357 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:09:27.826850 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 966/1894] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:09:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:09:27.830784 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c40edb-47c8-45b3-bd82-4dff43d5c810 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:27.843044 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c40edb-47c8-45b3-bd82-4dff43d5c810 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:27.846690 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88ffd08-3b79-4762-aa4b-831787ccfbb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:27.847375 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88ffd08-3b79-4762-aa4b-831787ccfbb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:27.847748 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88ffd08-3b79-4762-aa4b-831787ccfbb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:27.848027 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88ffd08-3b79-4762-aa4b-831787ccfbb2 admin admin] PATH_INFO: `/v3/projects/7f52c099475a4aee9f8bf097b4d7d711/users/12a65e4946e44230a82f2ba7027ac936/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:27.863726 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88ffd08-3b79-4762-aa4b-831787ccfbb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f52c099475a4aee9f8bf097b4d7d711, user_id=12a65e4946e44230a82f2ba7027ac936, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:27.866572 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88ffd08-3b79-4762-aa4b-831787ccfbb2 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:27.945547 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 929/1895] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:09:27 2022] PUT /identity/v3/projects/7f52c099475a4aee9f8bf097b4d7d711/users/12a65e4946e44230a82f2ba7027ac936/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:09:27.950113 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8249ca32-8186-4723-b847-591c5f83c574 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:27.960078 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8249ca32-8186-4723-b847-591c5f83c574 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:28.027018 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8bc0f5-939b-4275-bd60-0b7f6270713a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:28.028227 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8bc0f5-939b-4275-bd60-0b7f6270713a admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:28.028505 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8bc0f5-939b-4275-bd60-0b7f6270713a admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:28.028777 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8bc0f5-939b-4275-bd60-0b7f6270713a admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:28.029421 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8bc0f5-939b-4275-bd60-0b7f6270713a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:28.031058 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8bc0f5-939b-4275-bd60-0b7f6270713a admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:28.040703 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 967/1896] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:09:27 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:09:28.044896 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88ffd08-3b79-4762-aa4b-831787ccfbb2 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:28.055002 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88ffd08-3b79-4762-aa4b-831787ccfbb2 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:28.060035 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f4f894-a1bd-4011-8efe-044b52ec8032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:28.060676 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f4f894-a1bd-4011-8efe-044b52ec8032 admin admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:28.060952 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f4f894-a1bd-4011-8efe-044b52ec8032 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:28.061224 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f4f894-a1bd-4011-8efe-044b52ec8032 admin admin] PATH_INFO: `/v3/projects/7f52c099475a4aee9f8bf097b4d7d711/users/12a65e4946e44230a82f2ba7027ac936/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:28.063133 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f4f894-a1bd-4011-8efe-044b52ec8032 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f52c099475a4aee9f8bf097b4d7d711, user_id=12a65e4946e44230a82f2ba7027ac936, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:28.065752 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f4f894-a1bd-4011-8efe-044b52ec8032 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:28.076252 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 930/1897] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:09:28 2022] PUT /identity/v3/projects/7f52c099475a4aee9f8bf097b4d7d711/users/12a65e4946e44230a82f2ba7027ac936/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:09:28.086124 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861c6253-c61c-4827-acd0-d022a5b4bcd5 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:28.086415 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861c6253-c61c-4827-acd0-d022a5b4bcd5 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:28.086693 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861c6253-c61c-4827-acd0-d022a5b4bcd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:28.114022 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-861c6253-c61c-4827-acd0-d022a5b4bcd5 None None] MFA Rules not processed for user `12a65e4946e44230a82f2ba7027ac936`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:09:28.115237 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-861c6253-c61c-4827-acd0-d022a5b4bcd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:09:28.196569 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 968/1898] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:09:28 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:09:28.211697 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f4f894-a1bd-4011-8efe-044b52ec8032 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:28.213722 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-44f4f894-a1bd-4011-8efe-044b52ec8032 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:09:28.275113 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f4f894-a1bd-4011-8efe-044b52ec8032 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:28.279865 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51c425e-eb39-41fe-93c6-7a5c7672a2a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab599d98b0624a2e89090312bcab713f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:28.280471 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51c425e-eb39-41fe-93c6-7a5c7672a2a2 service cinder] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:28.280757 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51c425e-eb39-41fe-93c6-7a5c7672a2a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:28.281174 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51c425e-eb39-41fe-93c6-7a5c7672a2a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:28.290904 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51c425e-eb39-41fe-93c6-7a5c7672a2a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:28.292334 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51c425e-eb39-41fe-93c6-7a5c7672a2a2 service cinder] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:28.302054 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 931/1899] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:09:28 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:09:29.233119 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861c6253-c61c-4827-acd0-d022a5b4bcd5 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:29.242436 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861c6253-c61c-4827-acd0-d022a5b4bcd5 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:29.247473 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffae3a73-62fb-4213-b5b7-f58fcd4851b1 tempest-VolumeQuotaClassesTest-1825349178 tempest-VolumeQuotaClassesTest-1825349178-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a65e4946e44230a82f2ba7027ac936', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f52c099475a4aee9f8bf097b4d7d711', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:29.248109 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae3a73-62fb-4213-b5b7-f58fcd4851b1 tempest-VolumeQuotaClassesTest-1825349178 tempest-VolumeQuotaClassesTest-1825349178-project-admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:29.248456 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae3a73-62fb-4213-b5b7-f58fcd4851b1 tempest-VolumeQuotaClassesTest-1825349178 tempest-VolumeQuotaClassesTest-1825349178-project-admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:29.248896 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae3a73-62fb-4213-b5b7-f58fcd4851b1 tempest-VolumeQuotaClassesTest-1825349178 tempest-VolumeQuotaClassesTest-1825349178-project-admin] PATH_INFO: `/v3/domains` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:29.249744 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffae3a73-62fb-4213-b5b7-f58fcd4851b1 tempest-VolumeQuotaClassesTest-1825349178 tempest-VolumeQuotaClassesTest-1825349178-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 03:09:29.250151 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffae3a73-62fb-4213-b5b7-f58fcd4851b1 tempest-VolumeQuotaClassesTest-1825349178 tempest-VolumeQuotaClassesTest-1825349178-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:29.251871 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffae3a73-62fb-4213-b5b7-f58fcd4851b1 tempest-VolumeQuotaClassesTest-1825349178 tempest-VolumeQuotaClassesTest-1825349178-project-admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:29.276612 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 969/1900] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Oct 14 03:09:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 03:09:29.276746 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51c425e-eb39-41fe-93c6-7a5c7672a2a2 service cinder] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:29.279911 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51c425e-eb39-41fe-93c6-7a5c7672a2a2 service cinder] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:29.284515 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bd127c-0cc9-49a6-b666-3a0c931b1c0e tempest-VolumeQuotaClassesTest-1825349178 tempest-VolumeQuotaClassesTest-1825349178-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a65e4946e44230a82f2ba7027ac936', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f52c099475a4aee9f8bf097b4d7d711', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:29.285123 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd127c-0cc9-49a6-b666-3a0c931b1c0e tempest-VolumeQuotaClassesTest-1825349178 tempest-VolumeQuotaClassesTest-1825349178-project-admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:29.285403 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd127c-0cc9-49a6-b666-3a0c931b1c0e tempest-VolumeQuotaClassesTest-1825349178 tempest-VolumeQuotaClassesTest-1825349178-project-admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:29.285675 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd127c-0cc9-49a6-b666-3a0c931b1c0e tempest-VolumeQuotaClassesTest-1825349178 tempest-VolumeQuotaClassesTest-1825349178-project-admin] PATH_INFO: `/v3/projects` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:29.286354 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bd127c-0cc9-49a6-b666-3a0c931b1c0e tempest-VolumeQuotaClassesTest-1825349178 tempest-VolumeQuotaClassesTest-1825349178-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:29.288296 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bd127c-0cc9-49a6-b666-3a0c931b1c0e tempest-VolumeQuotaClassesTest-1825349178 tempest-VolumeQuotaClassesTest-1825349178-project-admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:29.369495 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 932/1901] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:09:29 2022] POST /identity/v3/projects => generated 355 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:09:29.373554 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffae3a73-62fb-4213-b5b7-f58fcd4851b1 tempest-VolumeQuotaClassesTest-1825349178 tempest-VolumeQuotaClassesTest-1825349178-project-admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:29.383638 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffae3a73-62fb-4213-b5b7-f58fcd4851b1 tempest-VolumeQuotaClassesTest-1825349178 tempest-VolumeQuotaClassesTest-1825349178-project-admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:29.442771 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88db2ab-e761-4d6a-a6e4-fa9059eec484 tempest-VolumeQuotaClassesTest-1825349178 tempest-VolumeQuotaClassesTest-1825349178-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a65e4946e44230a82f2ba7027ac936', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f52c099475a4aee9f8bf097b4d7d711', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:29.443380 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88db2ab-e761-4d6a-a6e4-fa9059eec484 tempest-VolumeQuotaClassesTest-1825349178 tempest-VolumeQuotaClassesTest-1825349178-project-admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:29.443663 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88db2ab-e761-4d6a-a6e4-fa9059eec484 tempest-VolumeQuotaClassesTest-1825349178 tempest-VolumeQuotaClassesTest-1825349178-project-admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:29.443938 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88db2ab-e761-4d6a-a6e4-fa9059eec484 tempest-VolumeQuotaClassesTest-1825349178 tempest-VolumeQuotaClassesTest-1825349178-project-admin] PATH_INFO: `/v3/domains` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:29.444582 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88db2ab-e761-4d6a-a6e4-fa9059eec484 tempest-VolumeQuotaClassesTest-1825349178 tempest-VolumeQuotaClassesTest-1825349178-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 03:09:29.444887 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88db2ab-e761-4d6a-a6e4-fa9059eec484 tempest-VolumeQuotaClassesTest-1825349178 tempest-VolumeQuotaClassesTest-1825349178-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:29.446529 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88db2ab-e761-4d6a-a6e4-fa9059eec484 tempest-VolumeQuotaClassesTest-1825349178 tempest-VolumeQuotaClassesTest-1825349178-project-admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:29.455297 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:09:29.459012 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 970/1902] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Oct 14 03:09:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 03:09:29.497133 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bd127c-0cc9-49a6-b666-3a0c931b1c0e tempest-VolumeQuotaClassesTest-1825349178 tempest-VolumeQuotaClassesTest-1825349178-project-admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:29.507396 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bd127c-0cc9-49a6-b666-3a0c931b1c0e tempest-VolumeQuotaClassesTest-1825349178 tempest-VolumeQuotaClassesTest-1825349178-project-admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:29.512395 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adacd0d3-3dcd-4d6c-ba19-8b405734529c tempest-VolumeQuotaClassesTest-1825349178 tempest-VolumeQuotaClassesTest-1825349178-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12a65e4946e44230a82f2ba7027ac936', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f52c099475a4aee9f8bf097b4d7d711', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:29.513011 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adacd0d3-3dcd-4d6c-ba19-8b405734529c tempest-VolumeQuotaClassesTest-1825349178 tempest-VolumeQuotaClassesTest-1825349178-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:29.513292 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adacd0d3-3dcd-4d6c-ba19-8b405734529c tempest-VolumeQuotaClassesTest-1825349178 tempest-VolumeQuotaClassesTest-1825349178-project-admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:29.513562 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adacd0d3-3dcd-4d6c-ba19-8b405734529c tempest-VolumeQuotaClassesTest-1825349178 tempest-VolumeQuotaClassesTest-1825349178-project-admin] PATH_INFO: `/v3/projects/603a0d3f4ed84dad87deb75eab10b9e3` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:29.514698 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adacd0d3-3dcd-4d6c-ba19-8b405734529c tempest-VolumeQuotaClassesTest-1825349178 tempest-VolumeQuotaClassesTest-1825349178-project-admin] RBAC: Authorizing `identity:delete_project(project_id=603a0d3f4ed84dad87deb75eab10b9e3)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:29.516589 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adacd0d3-3dcd-4d6c-ba19-8b405734529c tempest-VolumeQuotaClassesTest-1825349178 tempest-VolumeQuotaClassesTest-1825349178-project-admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:29.639328 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-adacd0d3-3dcd-4d6c-ba19-8b405734529c tempest-VolumeQuotaClassesTest-1825349178 tempest-VolumeQuotaClassesTest-1825349178-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '603a0d3f4ed84dad87deb75eab10b9e3', 'request_id': 'req-adacd0d3-3dcd-4d6c-ba19-8b405734529c'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:29.645989 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-adacd0d3-3dcd-4d6c-ba19-8b405734529c tempest-VolumeQuotaClassesTest-1825349178 tempest-VolumeQuotaClassesTest-1825349178-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '603a0d3f4ed84dad87deb75eab10b9e3', 'request_id': 'req-adacd0d3-3dcd-4d6c-ba19-8b405734529c'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:29.717217 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-adacd0d3-3dcd-4d6c-ba19-8b405734529c tempest-VolumeQuotaClassesTest-1825349178 tempest-VolumeQuotaClassesTest-1825349178-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '603a0d3f4ed84dad87deb75eab10b9e3', 'request_id': 'req-adacd0d3-3dcd-4d6c-ba19-8b405734529c'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:29.728191 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-adacd0d3-3dcd-4d6c-ba19-8b405734529c tempest-VolumeQuotaClassesTest-1825349178 tempest-VolumeQuotaClassesTest-1825349178-project-admin] The token cache is being invalidate because project 603a0d3f4ed84dad87deb75eab10b9e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:09:29.728490 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-adacd0d3-3dcd-4d6c-ba19-8b405734529c tempest-VolumeQuotaClassesTest-1825349178 tempest-VolumeQuotaClassesTest-1825349178-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:29.729128 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-adacd0d3-3dcd-4d6c-ba19-8b405734529c tempest-VolumeQuotaClassesTest-1825349178 tempest-VolumeQuotaClassesTest-1825349178-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:29.730675 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 933/1903] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:09:29 2022] DELETE /identity/v3/projects/603a0d3f4ed84dad87deb75eab10b9e3 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:09:30.896204 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88db2ab-e761-4d6a-a6e4-fa9059eec484 tempest-VolumeQuotaClassesTest-1825349178 tempest-VolumeQuotaClassesTest-1825349178-project-admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:30.898044 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-e88db2ab-e761-4d6a-a6e4-fa9059eec484 tempest-VolumeQuotaClassesTest-1825349178 tempest-VolumeQuotaClassesTest-1825349178-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:09:30.966162 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88db2ab-e761-4d6a-a6e4-fa9059eec484 tempest-VolumeQuotaClassesTest-1825349178 tempest-VolumeQuotaClassesTest-1825349178-project-admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:31.055336 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e845f28b-9091-41b3-9251-aeef03889c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:31.056105 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e845f28b-9091-41b3-9251-aeef03889c69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:31.056994 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e845f28b-9091-41b3-9251-aeef03889c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:31.057284 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e845f28b-9091-41b3-9251-aeef03889c69 admin admin] PATH_INFO: `/v3/users/8fea54e099f449c1887e34af8668ebfa` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:31.060005 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e845f28b-9091-41b3-9251-aeef03889c69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fea54e099f449c1887e34af8668ebfa)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:31.062176 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e845f28b-9091-41b3-9251-aeef03889c69 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:31.246937 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-e845f28b-9091-41b3-9251-aeef03889c69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fea54e099f449c1887e34af8668ebfa'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:31.247651 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-e845f28b-9091-41b3-9251-aeef03889c69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fea54e099f449c1887e34af8668ebfa'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:31.312042 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-e845f28b-9091-41b3-9251-aeef03889c69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fea54e099f449c1887e34af8668ebfa'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:31.312664 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-e845f28b-9091-41b3-9251-aeef03889c69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fea54e099f449c1887e34af8668ebfa'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:31.320742 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-e845f28b-9091-41b3-9251-aeef03889c69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fea54e099f449c1887e34af8668ebfa'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:31.338545 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 971/1904] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:09:30 2022] DELETE /identity/v3/users/8fea54e099f449c1887e34af8668ebfa => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:09:31.344516 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5647e2f-dbfb-4461-9889-eb74ce43277b None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:31.344516 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5647e2f-dbfb-4461-9889-eb74ce43277b None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:31.344516 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5647e2f-dbfb-4461-9889-eb74ce43277b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:31.365598 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-d5647e2f-dbfb-4461-9889-eb74ce43277b None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:09:31.367242 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-d5647e2f-dbfb-4461-9889-eb74ce43277b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:09:31.378744 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=86,keystone:INSERT=7,keystone:DELETE=7 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:09:31.438857 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 934/1905] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:09:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:09:31.445504 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e845f28b-9091-41b3-9251-aeef03889c69 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:31.447450 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-e845f28b-9091-41b3-9251-aeef03889c69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:09:31.481680 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:09:31.511217 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e845f28b-9091-41b3-9251-aeef03889c69 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:31.514891 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103710ba-ea4f-496c-8191-55fb5f09e4e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:31.515507 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103710ba-ea4f-496c-8191-55fb5f09e4e7 service neutron] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:31.515786 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103710ba-ea4f-496c-8191-55fb5f09e4e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:31.516086 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103710ba-ea4f-496c-8191-55fb5f09e4e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:31.527473 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103710ba-ea4f-496c-8191-55fb5f09e4e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:31.529024 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103710ba-ea4f-496c-8191-55fb5f09e4e7 service neutron] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:31.539670 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 972/1906] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:09:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:09:31.947436 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5647e2f-dbfb-4461-9889-eb74ce43277b None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:31.953784 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-d5647e2f-dbfb-4461-9889-eb74ce43277b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:09:31.961469 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5647e2f-dbfb-4461-9889-eb74ce43277b None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:31.967153 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853902ae-7794-4f0d-8086-3e7d59ccd1cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:31.968328 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853902ae-7794-4f0d-8086-3e7d59ccd1cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:31.979521 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853902ae-7794-4f0d-8086-3e7d59ccd1cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:31.984523 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853902ae-7794-4f0d-8086-3e7d59ccd1cf admin admin] PATH_INFO: `/v3/projects/fb434417cb0a427eb1fdbfde98c5de51` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:31.984523 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853902ae-7794-4f0d-8086-3e7d59ccd1cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb434417cb0a427eb1fdbfde98c5de51)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:31.984523 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853902ae-7794-4f0d-8086-3e7d59ccd1cf admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:32.134508 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-853902ae-7794-4f0d-8086-3e7d59ccd1cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb434417cb0a427eb1fdbfde98c5de51', 'request_id': 'req-853902ae-7794-4f0d-8086-3e7d59ccd1cf'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:32.141495 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-853902ae-7794-4f0d-8086-3e7d59ccd1cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb434417cb0a427eb1fdbfde98c5de51', 'request_id': 'req-853902ae-7794-4f0d-8086-3e7d59ccd1cf'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:32.300474 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-853902ae-7794-4f0d-8086-3e7d59ccd1cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb434417cb0a427eb1fdbfde98c5de51', 'request_id': 'req-853902ae-7794-4f0d-8086-3e7d59ccd1cf'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:32.313053 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-853902ae-7794-4f0d-8086-3e7d59ccd1cf admin admin] The token cache is being invalidate because project fb434417cb0a427eb1fdbfde98c5de51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:09:32.313354 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-853902ae-7794-4f0d-8086-3e7d59ccd1cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:32.314239 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-853902ae-7794-4f0d-8086-3e7d59ccd1cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:32.315956 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 935/1907] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:09:31 2022] DELETE /identity/v3/projects/fb434417cb0a427eb1fdbfde98c5de51 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:09:32.320868 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103710ba-ea4f-496c-8191-55fb5f09e4e7 service neutron] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:32.331855 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103710ba-ea4f-496c-8191-55fb5f09e4e7 service neutron] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:32.393214 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7e5fd4-0faf-4167-b410-06933485abbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:32.393889 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e5fd4-0faf-4167-b410-06933485abbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:32.394178 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e5fd4-0faf-4167-b410-06933485abbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:32.394454 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e5fd4-0faf-4167-b410-06933485abbc admin admin] PATH_INFO: `/v3/users/12a65e4946e44230a82f2ba7027ac936` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:32.395649 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7e5fd4-0faf-4167-b410-06933485abbc admin admin] RBAC: Authorizing `identity:delete_user(user_id=12a65e4946e44230a82f2ba7027ac936)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:32.397801 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7e5fd4-0faf-4167-b410-06933485abbc admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:32.627569 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-0e7e5fd4-0faf-4167-b410-06933485abbc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12a65e4946e44230a82f2ba7027ac936'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:32.628360 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-0e7e5fd4-0faf-4167-b410-06933485abbc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12a65e4946e44230a82f2ba7027ac936'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:32.734108 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-0e7e5fd4-0faf-4167-b410-06933485abbc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12a65e4946e44230a82f2ba7027ac936'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:32.734803 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-0e7e5fd4-0faf-4167-b410-06933485abbc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12a65e4946e44230a82f2ba7027ac936'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:32.742600 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-0e7e5fd4-0faf-4167-b410-06933485abbc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12a65e4946e44230a82f2ba7027ac936'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:32.767112 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 973/1908] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:09:32 2022] DELETE /identity/v3/users/12a65e4946e44230a82f2ba7027ac936 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:09:33.171352 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853902ae-7794-4f0d-8086-3e7d59ccd1cf admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:33.173044 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-853902ae-7794-4f0d-8086-3e7d59ccd1cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:09:33.186409 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853902ae-7794-4f0d-8086-3e7d59ccd1cf admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:33.267012 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c6083e-4881-48c9-b056-491bc6e67aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:33.267662 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c6083e-4881-48c9-b056-491bc6e67aa9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:33.268105 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c6083e-4881-48c9-b056-491bc6e67aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:33.268390 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c6083e-4881-48c9-b056-491bc6e67aa9 admin admin] PATH_INFO: `/v3/projects/7f52c099475a4aee9f8bf097b4d7d711` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:33.269534 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c6083e-4881-48c9-b056-491bc6e67aa9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f52c099475a4aee9f8bf097b4d7d711)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:33.271639 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c6083e-4881-48c9-b056-491bc6e67aa9 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:33.387949 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:09:33.396858 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-c9c6083e-4881-48c9-b056-491bc6e67aa9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f52c099475a4aee9f8bf097b4d7d711', 'request_id': 'req-c9c6083e-4881-48c9-b056-491bc6e67aa9'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:33.403192 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-c9c6083e-4881-48c9-b056-491bc6e67aa9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f52c099475a4aee9f8bf097b4d7d711', 'request_id': 'req-c9c6083e-4881-48c9-b056-491bc6e67aa9'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:33.489338 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-c9c6083e-4881-48c9-b056-491bc6e67aa9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f52c099475a4aee9f8bf097b4d7d711', 'request_id': 'req-c9c6083e-4881-48c9-b056-491bc6e67aa9'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:33.502596 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-c9c6083e-4881-48c9-b056-491bc6e67aa9 admin admin] The token cache is being invalidate because project 7f52c099475a4aee9f8bf097b4d7d711 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:09:33.502891 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-c9c6083e-4881-48c9-b056-491bc6e67aa9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:33.503601 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-c9c6083e-4881-48c9-b056-491bc6e67aa9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:33.505228 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 936/1909] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:09:33 2022] DELETE /identity/v3/projects/7f52c099475a4aee9f8bf097b4d7d711 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:09:33.510290 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2667df5a-d4d6-40a2-9561-062aecef26fc None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:33.510778 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2667df5a-d4d6-40a2-9561-062aecef26fc None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:33.510866 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2667df5a-d4d6-40a2-9561-062aecef26fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:33.531817 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-2667df5a-d4d6-40a2-9561-062aecef26fc None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:09:33.533045 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-2667df5a-d4d6-40a2-9561-062aecef26fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:09:33.579014 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:09:33.594081 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 974/1910] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:09:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:09:33.601917 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8b048b-3820-444c-9a3f-9df803ae5f39 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:33.602234 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8b048b-3820-444c-9a3f-9df803ae5f39 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:33.602515 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8b048b-3820-444c-9a3f-9df803ae5f39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:33.633829 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-eb8b048b-3820-444c-9a3f-9df803ae5f39 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:09:33.635407 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-eb8b048b-3820-444c-9a3f-9df803ae5f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:09:33.644363 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 937/1911] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:09:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:09:33.653182 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2667df5a-d4d6-40a2-9561-062aecef26fc None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:33.658814 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2667df5a-d4d6-40a2-9561-062aecef26fc None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:33.663510 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5fcb97-5558-4734-9567-5df6d481a901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:33.664111 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5fcb97-5558-4734-9567-5df6d481a901 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:33.664406 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5fcb97-5558-4734-9567-5df6d481a901 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:33.664735 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5fcb97-5558-4734-9567-5df6d481a901 admin admin] PATH_INFO: `/v3/domains` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:33.665315 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5fcb97-5558-4734-9567-5df6d481a901 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 03:09:33.665648 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5fcb97-5558-4734-9567-5df6d481a901 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:33.667352 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5fcb97-5558-4734-9567-5df6d481a901 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:33.683967 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 975/1912] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Oct 14 03:09:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 03:09:33.689447 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8b048b-3820-444c-9a3f-9df803ae5f39 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:33.694905 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8b048b-3820-444c-9a3f-9df803ae5f39 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:33.699531 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dec4b7-462c-45d1-9fd8-1f7c677a7a89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:33.700183 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dec4b7-462c-45d1-9fd8-1f7c677a7a89 admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:33.700468 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dec4b7-462c-45d1-9fd8-1f7c677a7a89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:33.700740 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dec4b7-462c-45d1-9fd8-1f7c677a7a89 admin admin] PATH_INFO: `/v3/projects` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:33.701416 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dec4b7-462c-45d1-9fd8-1f7c677a7a89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:33.703425 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dec4b7-462c-45d1-9fd8-1f7c677a7a89 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:33.817067 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 938/1913] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:09:33 2022] POST /identity/v3/projects => generated 383 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:09:33.819714 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5fcb97-5558-4734-9567-5df6d481a901 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:33.829686 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5fcb97-5558-4734-9567-5df6d481a901 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:33.897809 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573cfdf8-4680-4398-86f1-02d00d98b7ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:33.898423 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573cfdf8-4680-4398-86f1-02d00d98b7ce admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:33.898705 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573cfdf8-4680-4398-86f1-02d00d98b7ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:33.898977 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573cfdf8-4680-4398-86f1-02d00d98b7ce admin admin] PATH_INFO: `/v3/users` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:33.899648 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573cfdf8-4680-4398-86f1-02d00d98b7ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:33.902478 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573cfdf8-4680-4398-86f1-02d00d98b7ce admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:34.032065 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 976/1914] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:09:33 2022] POST /identity/v3/users => generated 292 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:09:34.033790 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dec4b7-462c-45d1-9fd8-1f7c677a7a89 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:34.045320 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dec4b7-462c-45d1-9fd8-1f7c677a7a89 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:34.050504 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768d89a4-e8ec-4dd5-bcdf-0dee13738677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:34.051119 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768d89a4-e8ec-4dd5-bcdf-0dee13738677 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:34.051402 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768d89a4-e8ec-4dd5-bcdf-0dee13738677 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:34.051711 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768d89a4-e8ec-4dd5-bcdf-0dee13738677 admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:34.052329 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768d89a4-e8ec-4dd5-bcdf-0dee13738677 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:34.054000 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768d89a4-e8ec-4dd5-bcdf-0dee13738677 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:34.064539 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 939/1915] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:09:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:09:34.068644 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573cfdf8-4680-4398-86f1-02d00d98b7ce admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:34.079039 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573cfdf8-4680-4398-86f1-02d00d98b7ce admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:34.086061 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822b3f6a-08c7-4cec-a58e-dfdf9e0d7c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:34.086710 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b3f6a-08c7-4cec-a58e-dfdf9e0d7c16 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:34.086992 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b3f6a-08c7-4cec-a58e-dfdf9e0d7c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:34.090229 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b3f6a-08c7-4cec-a58e-dfdf9e0d7c16 admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:34.090229 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822b3f6a-08c7-4cec-a58e-dfdf9e0d7c16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:34.090229 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822b3f6a-08c7-4cec-a58e-dfdf9e0d7c16 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:34.107134 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 977/1916] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:09:34 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:09:34.111969 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768d89a4-e8ec-4dd5-bcdf-0dee13738677 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:34.123641 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768d89a4-e8ec-4dd5-bcdf-0dee13738677 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:34.134087 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce94ec9-6fec-4673-a4d0-12d4feedcb4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:34.134751 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce94ec9-6fec-4673-a4d0-12d4feedcb4a admin admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:34.135031 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce94ec9-6fec-4673-a4d0-12d4feedcb4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:34.135305 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce94ec9-6fec-4673-a4d0-12d4feedcb4a admin admin] PATH_INFO: `/v3/projects/f201de3b00864b90967a2e260622f07a/users/fba8fef0a2a248e3b2a56a0385f29caa/roles/9bd0470b544240209a420f640c79c06c` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:34.150882 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce94ec9-6fec-4673-a4d0-12d4feedcb4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f201de3b00864b90967a2e260622f07a, user_id=fba8fef0a2a248e3b2a56a0385f29caa, role_id=9bd0470b544240209a420f640c79c06c)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:34.154119 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce94ec9-6fec-4673-a4d0-12d4feedcb4a admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:34.225259 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 940/1917] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:09:34 2022] PUT /identity/v3/projects/f201de3b00864b90967a2e260622f07a/users/fba8fef0a2a248e3b2a56a0385f29caa/roles/9bd0470b544240209a420f640c79c06c => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:09:34.243546 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb08cf60-194c-4e75-a3c8-ae71aca380e0 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:34.243546 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb08cf60-194c-4e75-a3c8-ae71aca380e0 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:34.243872 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb08cf60-194c-4e75-a3c8-ae71aca380e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:34.272397 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-fb08cf60-194c-4e75-a3c8-ae71aca380e0 None None] MFA Rules not processed for user `fba8fef0a2a248e3b2a56a0385f29caa`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:09:34.273615 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-fb08cf60-194c-4e75-a3c8-ae71aca380e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:09:34.337752 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 978/1918] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:09:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:09:34.344838 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce94ec9-6fec-4673-a4d0-12d4feedcb4a admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:34.411457 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce94ec9-6fec-4673-a4d0-12d4feedcb4a admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:34.416298 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee00dc14-f75a-48fb-8439-4a055b642135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:34.416907 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee00dc14-f75a-48fb-8439-4a055b642135 admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:34.417191 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee00dc14-f75a-48fb-8439-4a055b642135 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:34.417470 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee00dc14-f75a-48fb-8439-4a055b642135 admin admin] PATH_INFO: `/v3/projects` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:34.418142 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee00dc14-f75a-48fb-8439-4a055b642135 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:34.420086 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee00dc14-f75a-48fb-8439-4a055b642135 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:34.488059 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 941/1919] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:09:34 2022] POST /identity/v3/projects => generated 385 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:09:34.489402 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb08cf60-194c-4e75-a3c8-ae71aca380e0 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:34.501144 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb08cf60-194c-4e75-a3c8-ae71aca380e0 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:34.546268 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:09:34.561058 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6727912-7114-4902-bd32-b7f8c462b110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:34.561667 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6727912-7114-4902-bd32-b7f8c462b110 admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:34.561974 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6727912-7114-4902-bd32-b7f8c462b110 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:34.562244 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6727912-7114-4902-bd32-b7f8c462b110 admin admin] PATH_INFO: `/v3/users` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:34.562884 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6727912-7114-4902-bd32-b7f8c462b110 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:34.564787 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6727912-7114-4902-bd32-b7f8c462b110 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:34.672600 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 979/1920] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:09:34 2022] POST /identity/v3/users => generated 293 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:09:34.676494 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee00dc14-f75a-48fb-8439-4a055b642135 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:34.686744 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee00dc14-f75a-48fb-8439-4a055b642135 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:34.691445 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a497adf1-e9b3-4c1e-99a6-0d21b990f49c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:34.692050 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a497adf1-e9b3-4c1e-99a6-0d21b990f49c admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:34.692335 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a497adf1-e9b3-4c1e-99a6-0d21b990f49c admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:34.692610 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a497adf1-e9b3-4c1e-99a6-0d21b990f49c admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:34.693261 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a497adf1-e9b3-4c1e-99a6-0d21b990f49c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:34.695121 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a497adf1-e9b3-4c1e-99a6-0d21b990f49c admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:34.720713 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 942/1921] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:09:34 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:09:34.724481 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6727912-7114-4902-bd32-b7f8c462b110 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:34.735012 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6727912-7114-4902-bd32-b7f8c462b110 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:34.739662 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821b7731-d9cf-42e6-b817-0477d8a65379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:34.740268 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821b7731-d9cf-42e6-b817-0477d8a65379 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:34.740541 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821b7731-d9cf-42e6-b817-0477d8a65379 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:34.740814 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821b7731-d9cf-42e6-b817-0477d8a65379 admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:34.741484 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821b7731-d9cf-42e6-b817-0477d8a65379 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:34.743116 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821b7731-d9cf-42e6-b817-0477d8a65379 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:34.753943 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 980/1922] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:09:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:09:34.758080 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a497adf1-e9b3-4c1e-99a6-0d21b990f49c admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:34.768668 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a497adf1-e9b3-4c1e-99a6-0d21b990f49c admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:34.773184 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63967368-4c26-4f0f-a2bd-18afdd9912ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:34.773866 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63967368-4c26-4f0f-a2bd-18afdd9912ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:34.774145 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63967368-4c26-4f0f-a2bd-18afdd9912ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:34.774415 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63967368-4c26-4f0f-a2bd-18afdd9912ec admin admin] PATH_INFO: `/v3/projects/3422fcd443ba4d2193c72a60472adead/users/9f918ff2a5ff44dda2783e74c7701521/roles/9bd0470b544240209a420f640c79c06c` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:34.789580 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63967368-4c26-4f0f-a2bd-18afdd9912ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=3422fcd443ba4d2193c72a60472adead, user_id=9f918ff2a5ff44dda2783e74c7701521, role_id=9bd0470b544240209a420f640c79c06c)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:34.792330 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63967368-4c26-4f0f-a2bd-18afdd9912ec admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:34.872688 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 943/1923] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:09:34 2022] PUT /identity/v3/projects/3422fcd443ba4d2193c72a60472adead/users/9f918ff2a5ff44dda2783e74c7701521/roles/9bd0470b544240209a420f640c79c06c => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:09:34.883109 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56259356-c89e-4a42-a5fa-bd77343ad70a None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:34.883400 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56259356-c89e-4a42-a5fa-bd77343ad70a None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:34.883675 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56259356-c89e-4a42-a5fa-bd77343ad70a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:34.912113 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-56259356-c89e-4a42-a5fa-bd77343ad70a None None] MFA Rules not processed for user `9f918ff2a5ff44dda2783e74c7701521`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:09:34.913358 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-56259356-c89e-4a42-a5fa-bd77343ad70a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:09:34.977092 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 981/1924] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:09:34 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:09:34.981576 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63967368-4c26-4f0f-a2bd-18afdd9912ec admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:34.994953 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:09:35.047506 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63967368-4c26-4f0f-a2bd-18afdd9912ec admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:35.052133 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9842ef0-4e9e-41b5-8321-1eebab8507b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:35.052746 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9842ef0-4e9e-41b5-8321-1eebab8507b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:35.053044 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9842ef0-4e9e-41b5-8321-1eebab8507b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:35.053526 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9842ef0-4e9e-41b5-8321-1eebab8507b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:35.054431 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9842ef0-4e9e-41b5-8321-1eebab8507b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:35.056750 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9842ef0-4e9e-41b5-8321-1eebab8507b8 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:35.188499 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 944/1925] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:09:34 2022] POST /identity/v3/projects => generated 385 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 03:09:35.192404 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56259356-c89e-4a42-a5fa-bd77343ad70a None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:35.203143 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56259356-c89e-4a42-a5fa-bd77343ad70a None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:35.264424 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c111d03-ebee-409a-9241-b8e2ef2579e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:35.265241 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c111d03-ebee-409a-9241-b8e2ef2579e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:35.265480 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c111d03-ebee-409a-9241-b8e2ef2579e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:35.265777 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c111d03-ebee-409a-9241-b8e2ef2579e1 admin admin] PATH_INFO: `/v3/users` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:35.266579 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c111d03-ebee-409a-9241-b8e2ef2579e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:35.268507 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c111d03-ebee-409a-9241-b8e2ef2579e1 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:35.368548 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 982/1926] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:09:35 2022] POST /identity/v3/users => generated 299 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:09:35.373215 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9842ef0-4e9e-41b5-8321-1eebab8507b8 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:35.383805 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9842ef0-4e9e-41b5-8321-1eebab8507b8 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:35.388832 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3731f0-c72d-4f02-8e9b-ce4c69792b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:35.389465 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3731f0-c72d-4f02-8e9b-ce4c69792b95 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:35.389748 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3731f0-c72d-4f02-8e9b-ce4c69792b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:35.390058 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3731f0-c72d-4f02-8e9b-ce4c69792b95 admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:35.390728 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3731f0-c72d-4f02-8e9b-ce4c69792b95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:35.392393 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3731f0-c72d-4f02-8e9b-ce4c69792b95 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:35.403631 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 945/1927] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:09:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:09:35.407816 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c111d03-ebee-409a-9241-b8e2ef2579e1 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:35.417658 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c111d03-ebee-409a-9241-b8e2ef2579e1 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:35.422672 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec502024-7523-4f56-865d-943299ff72c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:35.423312 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec502024-7523-4f56-865d-943299ff72c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:35.423590 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec502024-7523-4f56-865d-943299ff72c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:35.423860 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec502024-7523-4f56-865d-943299ff72c2 admin admin] PATH_INFO: `/v3/projects/ca23062d165c4b72ab4606722559e51e/users/f8e9e1907ddf44e1ae946b81703eee10/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:35.438783 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec502024-7523-4f56-865d-943299ff72c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca23062d165c4b72ab4606722559e51e, user_id=f8e9e1907ddf44e1ae946b81703eee10, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:35.441416 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec502024-7523-4f56-865d-943299ff72c2 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:35.511596 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 983/1928] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:09:35 2022] PUT /identity/v3/projects/ca23062d165c4b72ab4606722559e51e/users/f8e9e1907ddf44e1ae946b81703eee10/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:09:35.515597 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3731f0-c72d-4f02-8e9b-ce4c69792b95 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:35.525467 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3731f0-c72d-4f02-8e9b-ce4c69792b95 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:35.584986 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8af1d77-5f0d-45e4-a3d1-3a2d05c13fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:35.585595 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8af1d77-5f0d-45e4-a3d1-3a2d05c13fb5 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:35.585913 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8af1d77-5f0d-45e4-a3d1-3a2d05c13fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:35.586187 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8af1d77-5f0d-45e4-a3d1-3a2d05c13fb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:35.586823 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8af1d77-5f0d-45e4-a3d1-3a2d05c13fb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:35.588412 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8af1d77-5f0d-45e4-a3d1-3a2d05c13fb5 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:35.598894 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 946/1929] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:09:35 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:09:35.602866 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec502024-7523-4f56-865d-943299ff72c2 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:35.614292 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec502024-7523-4f56-865d-943299ff72c2 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:35.618840 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421f53c7-ccf3-46b1-af33-3e924e6ad010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:35.619486 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421f53c7-ccf3-46b1-af33-3e924e6ad010 admin admin] REQUEST_METHOD: `PUT` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:35.619760 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421f53c7-ccf3-46b1-af33-3e924e6ad010 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:35.620032 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421f53c7-ccf3-46b1-af33-3e924e6ad010 admin admin] PATH_INFO: `/v3/projects/ca23062d165c4b72ab4606722559e51e/users/f8e9e1907ddf44e1ae946b81703eee10/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:35.621989 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421f53c7-ccf3-46b1-af33-3e924e6ad010 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca23062d165c4b72ab4606722559e51e, user_id=f8e9e1907ddf44e1ae946b81703eee10, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:35.624627 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421f53c7-ccf3-46b1-af33-3e924e6ad010 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:35.634573 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 984/1930] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:09:35 2022] PUT /identity/v3/projects/ca23062d165c4b72ab4606722559e51e/users/f8e9e1907ddf44e1ae946b81703eee10/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:09:35.646736 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307108a0-b314-4a7a-9b45-50e381331f7d None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:35.647045 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307108a0-b314-4a7a-9b45-50e381331f7d None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:35.647320 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307108a0-b314-4a7a-9b45-50e381331f7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:35.677273 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-307108a0-b314-4a7a-9b45-50e381331f7d None None] MFA Rules not processed for user `f8e9e1907ddf44e1ae946b81703eee10`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:09:35.678543 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-307108a0-b314-4a7a-9b45-50e381331f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:09:35.755462 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 947/1931] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:09:35 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:09:35.762429 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421f53c7-ccf3-46b1-af33-3e924e6ad010 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:35.764026 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-421f53c7-ccf3-46b1-af33-3e924e6ad010 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:09:35.770433 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:09:35.828053 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421f53c7-ccf3-46b1-af33-3e924e6ad010 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:35.831811 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f585a016-3da4-4662-b941-d7c89020019f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab599d98b0624a2e89090312bcab713f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:35.832461 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f585a016-3da4-4662-b941-d7c89020019f service cinder] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:35.832744 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f585a016-3da4-4662-b941-d7c89020019f service cinder] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:35.833020 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f585a016-3da4-4662-b941-d7c89020019f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:35.890246 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f585a016-3da4-4662-b941-d7c89020019f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:35.891687 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f585a016-3da4-4662-b941-d7c89020019f service cinder] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:35.900506 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 985/1932] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:09:35 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:09:37.809806 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307108a0-b314-4a7a-9b45-50e381331f7d None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:37.811664 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-307108a0-b314-4a7a-9b45-50e381331f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:09:37.823346 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307108a0-b314-4a7a-9b45-50e381331f7d None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:37.827744 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eda7b4-8a10-404d-a7ff-b9a130c3fcb9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab599d98b0624a2e89090312bcab713f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:37.828347 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eda7b4-8a10-404d-a7ff-b9a130c3fcb9 service cinder] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:37.828655 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eda7b4-8a10-404d-a7ff-b9a130c3fcb9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:37.828930 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eda7b4-8a10-404d-a7ff-b9a130c3fcb9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:37.834127 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:09:37.886243 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eda7b4-8a10-404d-a7ff-b9a130c3fcb9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:37.887865 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eda7b4-8a10-404d-a7ff-b9a130c3fcb9 service cinder] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:37.898954 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 948/1933] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:09:37 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:09:39.456385 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f585a016-3da4-4662-b941-d7c89020019f service cinder] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:39.465157 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f585a016-3da4-4662-b941-d7c89020019f service cinder] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:39.468786 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69061868-de39-4e81-bd6d-063a2c94641d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab599d98b0624a2e89090312bcab713f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:39.469385 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69061868-de39-4e81-bd6d-063a2c94641d service cinder] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:39.469670 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69061868-de39-4e81-bd6d-063a2c94641d service cinder] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:39.469980 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69061868-de39-4e81-bd6d-063a2c94641d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:39.478941 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69061868-de39-4e81-bd6d-063a2c94641d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:39.480382 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69061868-de39-4e81-bd6d-063a2c94641d service cinder] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:39.490074 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 986/1934] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:09:39 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:09:44.510574 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f07b564-751c-4926-a704-e49d364cac98 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:44.510574 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f07b564-751c-4926-a704-e49d364cac98 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:44.510574 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f07b564-751c-4926-a704-e49d364cac98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:44.547855 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-0f07b564-751c-4926-a704-e49d364cac98 None None] MFA Rules not processed for user `410b7c9e93c94ce78d1b5c0e646307a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:09:44.549079 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-0f07b564-751c-4926-a704-e49d364cac98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:09:44.618353 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 949/1935] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:09:44 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:09:45.684550 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69061868-de39-4e81-bd6d-063a2c94641d service cinder] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:45.789029 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69061868-de39-4e81-bd6d-063a2c94641d service cinder] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:45.794291 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8db22b8-acc3-4b8c-b8ca-298c2fd70388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:45.795011 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8db22b8-acc3-4b8c-b8ca-298c2fd70388 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:45.795522 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8db22b8-acc3-4b8c-b8ca-298c2fd70388 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:45.795803 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8db22b8-acc3-4b8c-b8ca-298c2fd70388 admin admin] PATH_INFO: `/v3/users/fba8fef0a2a248e3b2a56a0385f29caa` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:45.796955 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8db22b8-acc3-4b8c-b8ca-298c2fd70388 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fba8fef0a2a248e3b2a56a0385f29caa)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:45.798933 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8db22b8-acc3-4b8c-b8ca-298c2fd70388 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:46.013834 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-b8db22b8-acc3-4b8c-b8ca-298c2fd70388 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fba8fef0a2a248e3b2a56a0385f29caa'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:46.014805 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-b8db22b8-acc3-4b8c-b8ca-298c2fd70388 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fba8fef0a2a248e3b2a56a0385f29caa'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:46.016994 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:09:46.088818 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-b8db22b8-acc3-4b8c-b8ca-298c2fd70388 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fba8fef0a2a248e3b2a56a0385f29caa'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:46.089960 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-b8db22b8-acc3-4b8c-b8ca-298c2fd70388 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fba8fef0a2a248e3b2a56a0385f29caa'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:46.097782 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-b8db22b8-acc3-4b8c-b8ca-298c2fd70388 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fba8fef0a2a248e3b2a56a0385f29caa'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:46.116183 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 987/1936] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:09:45 2022] DELETE /identity/v3/users/fba8fef0a2a248e3b2a56a0385f29caa => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:09:46.121024 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4b4ece-b624-4c7e-af0f-37b78a88ba95 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:46.121547 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4b4ece-b624-4c7e-af0f-37b78a88ba95 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:46.121689 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4b4ece-b624-4c7e-af0f-37b78a88ba95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:46.141981 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-ea4b4ece-b624-4c7e-af0f-37b78a88ba95 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:09:46.143182 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-ea4b4ece-b624-4c7e-af0f-37b78a88ba95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:09:46.205715 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 950/1937] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:09:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:09:46.215698 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8db22b8-acc3-4b8c-b8ca-298c2fd70388 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:46.217146 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-b8db22b8-acc3-4b8c-b8ca-298c2fd70388 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:09:46.278848 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8db22b8-acc3-4b8c-b8ca-298c2fd70388 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:46.282740 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67def5f0-3686-4441-a44d-3f7d1b39fd41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:46.286868 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67def5f0-3686-4441-a44d-3f7d1b39fd41 service neutron] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:46.286868 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67def5f0-3686-4441-a44d-3f7d1b39fd41 service neutron] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:46.286868 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67def5f0-3686-4441-a44d-3f7d1b39fd41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:46.292979 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67def5f0-3686-4441-a44d-3f7d1b39fd41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:46.294481 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67def5f0-3686-4441-a44d-3f7d1b39fd41 service neutron] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:46.304126 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 988/1938] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:09:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:09:46.804232 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4b4ece-b624-4c7e-af0f-37b78a88ba95 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:46.806260 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-ea4b4ece-b624-4c7e-af0f-37b78a88ba95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:09:46.832557 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4b4ece-b624-4c7e-af0f-37b78a88ba95 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:46.838075 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514e93c6-91c6-4d38-afad-f4de11e2b0b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:46.838882 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514e93c6-91c6-4d38-afad-f4de11e2b0b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:46.839578 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514e93c6-91c6-4d38-afad-f4de11e2b0b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:46.839965 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514e93c6-91c6-4d38-afad-f4de11e2b0b4 admin admin] PATH_INFO: `/v3/projects/f201de3b00864b90967a2e260622f07a` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:46.841652 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514e93c6-91c6-4d38-afad-f4de11e2b0b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f201de3b00864b90967a2e260622f07a)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:46.843789 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514e93c6-91c6-4d38-afad-f4de11e2b0b4 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:46.957953 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:09:46.959723 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-514e93c6-91c6-4d38-afad-f4de11e2b0b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f201de3b00864b90967a2e260622f07a', 'request_id': 'req-514e93c6-91c6-4d38-afad-f4de11e2b0b4'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:46.968564 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-514e93c6-91c6-4d38-afad-f4de11e2b0b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f201de3b00864b90967a2e260622f07a', 'request_id': 'req-514e93c6-91c6-4d38-afad-f4de11e2b0b4'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:47.065505 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-514e93c6-91c6-4d38-afad-f4de11e2b0b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f201de3b00864b90967a2e260622f07a', 'request_id': 'req-514e93c6-91c6-4d38-afad-f4de11e2b0b4'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:47.079751 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-514e93c6-91c6-4d38-afad-f4de11e2b0b4 admin admin] The token cache is being invalidate because project f201de3b00864b90967a2e260622f07a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:09:47.080045 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-514e93c6-91c6-4d38-afad-f4de11e2b0b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:47.080785 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-514e93c6-91c6-4d38-afad-f4de11e2b0b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:47.082524 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 951/1939] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:09:46 2022] DELETE /identity/v3/projects/f201de3b00864b90967a2e260622f07a => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:09:47.086574 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67def5f0-3686-4441-a44d-3f7d1b39fd41 service neutron] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:47.097316 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67def5f0-3686-4441-a44d-3f7d1b39fd41 service neutron] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:47.176754 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a060dd8-d117-435e-a004-6598a9f06118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:47.177407 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a060dd8-d117-435e-a004-6598a9f06118 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:47.177733 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a060dd8-d117-435e-a004-6598a9f06118 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:47.178138 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a060dd8-d117-435e-a004-6598a9f06118 admin admin] PATH_INFO: `/v3/users/9f918ff2a5ff44dda2783e74c7701521` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:47.179233 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a060dd8-d117-435e-a004-6598a9f06118 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f918ff2a5ff44dda2783e74c7701521)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:47.181240 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a060dd8-d117-435e-a004-6598a9f06118 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:47.381514 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-2a060dd8-d117-435e-a004-6598a9f06118 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f918ff2a5ff44dda2783e74c7701521'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:47.382200 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-2a060dd8-d117-435e-a004-6598a9f06118 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f918ff2a5ff44dda2783e74c7701521'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:47.494646 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-2a060dd8-d117-435e-a004-6598a9f06118 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f918ff2a5ff44dda2783e74c7701521'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:47.495259 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-2a060dd8-d117-435e-a004-6598a9f06118 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f918ff2a5ff44dda2783e74c7701521'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:47.503557 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-2a060dd8-d117-435e-a004-6598a9f06118 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f918ff2a5ff44dda2783e74c7701521'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:47.512237 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:INSERT=2,keystone:DELETE=12,keystone:SELECT=86 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:09:47.525546 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 989/1940] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:09:47 2022] DELETE /identity/v3/users/9f918ff2a5ff44dda2783e74c7701521 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:09:48.376000 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514e93c6-91c6-4d38-afad-f4de11e2b0b4 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:48.377712 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-514e93c6-91c6-4d38-afad-f4de11e2b0b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:09:48.409470 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514e93c6-91c6-4d38-afad-f4de11e2b0b4 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:48.482700 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6a3ddd-3dbe-4775-8130-c4f3cec1faea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:48.483367 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6a3ddd-3dbe-4775-8130-c4f3cec1faea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:48.483927 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6a3ddd-3dbe-4775-8130-c4f3cec1faea admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:48.484205 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6a3ddd-3dbe-4775-8130-c4f3cec1faea admin admin] PATH_INFO: `/v3/projects/3422fcd443ba4d2193c72a60472adead` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:48.485318 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6a3ddd-3dbe-4775-8130-c4f3cec1faea admin admin] RBAC: Authorizing `identity:delete_project(project_id=3422fcd443ba4d2193c72a60472adead)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:48.487487 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6a3ddd-3dbe-4775-8130-c4f3cec1faea admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:48.592809 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-8d6a3ddd-3dbe-4775-8130-c4f3cec1faea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3422fcd443ba4d2193c72a60472adead', 'request_id': 'req-8d6a3ddd-3dbe-4775-8130-c4f3cec1faea'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:48.599725 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-8d6a3ddd-3dbe-4775-8130-c4f3cec1faea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3422fcd443ba4d2193c72a60472adead', 'request_id': 'req-8d6a3ddd-3dbe-4775-8130-c4f3cec1faea'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:48.662189 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-8d6a3ddd-3dbe-4775-8130-c4f3cec1faea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3422fcd443ba4d2193c72a60472adead', 'request_id': 'req-8d6a3ddd-3dbe-4775-8130-c4f3cec1faea'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:48.689985 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-8d6a3ddd-3dbe-4775-8130-c4f3cec1faea admin admin] The token cache is being invalidate because project 3422fcd443ba4d2193c72a60472adead was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:09:48.697058 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-8d6a3ddd-3dbe-4775-8130-c4f3cec1faea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:48.697058 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-8d6a3ddd-3dbe-4775-8130-c4f3cec1faea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:48.698340 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a060dd8-d117-435e-a004-6598a9f06118 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:48.698835 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 952/1941] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:09:48 2022] DELETE /identity/v3/projects/3422fcd443ba4d2193c72a60472adead => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:09:48.701156 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-2a060dd8-d117-435e-a004-6598a9f06118 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:09:48.714798 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a060dd8-d117-435e-a004-6598a9f06118 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:48.760839 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbda39cb-53a6-4fb1-980e-90dc583cd708 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:48.760839 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbda39cb-53a6-4fb1-980e-90dc583cd708 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:48.761740 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbda39cb-53a6-4fb1-980e-90dc583cd708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:48.785383 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-dbda39cb-53a6-4fb1-980e-90dc583cd708 None None] MFA Rules not processed for user `7a4ea9e3987e4f0b9ad9ebe6d3c69b7b`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:09:48.786650 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-dbda39cb-53a6-4fb1-980e-90dc583cd708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:09:48.786838 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d7fd76-3a7d-4b7c-8224-9648393fd053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:48.787465 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d7fd76-3a7d-4b7c-8224-9648393fd053 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:48.787753 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d7fd76-3a7d-4b7c-8224-9648393fd053 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:48.788053 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d7fd76-3a7d-4b7c-8224-9648393fd053 admin admin] PATH_INFO: `/v3/users/f8e9e1907ddf44e1ae946b81703eee10` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:48.789191 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d7fd76-3a7d-4b7c-8224-9648393fd053 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8e9e1907ddf44e1ae946b81703eee10)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:48.791572 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d7fd76-3a7d-4b7c-8224-9648393fd053 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:48.851099 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 953/1942] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:09:48 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 03:09:49.080468 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-32d7fd76-3a7d-4b7c-8224-9648393fd053 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8e9e1907ddf44e1ae946b81703eee10'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:49.081212 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-32d7fd76-3a7d-4b7c-8224-9648393fd053 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8e9e1907ddf44e1ae946b81703eee10'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:49.163163 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-32d7fd76-3a7d-4b7c-8224-9648393fd053 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8e9e1907ddf44e1ae946b81703eee10'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:49.163873 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-32d7fd76-3a7d-4b7c-8224-9648393fd053 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8e9e1907ddf44e1ae946b81703eee10'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:49.175399 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-32d7fd76-3a7d-4b7c-8224-9648393fd053 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8e9e1907ddf44e1ae946b81703eee10'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:49.193196 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 990/1943] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:09:48 2022] DELETE /identity/v3/users/f8e9e1907ddf44e1ae946b81703eee10 => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:09:49.951106 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbda39cb-53a6-4fb1-980e-90dc583cd708 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:49.953078 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-dbda39cb-53a6-4fb1-980e-90dc583cd708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:09:49.968730 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbda39cb-53a6-4fb1-980e-90dc583cd708 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:49.978045 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:09:50.029674 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da710d4-b28f-4c94-94a4-e4307d551eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:50.030364 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da710d4-b28f-4c94-94a4-e4307d551eca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:50.030644 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da710d4-b28f-4c94-94a4-e4307d551eca admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:50.030915 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da710d4-b28f-4c94-94a4-e4307d551eca admin admin] PATH_INFO: `/v3/projects/ca23062d165c4b72ab4606722559e51e` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:50.032024 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da710d4-b28f-4c94-94a4-e4307d551eca admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca23062d165c4b72ab4606722559e51e)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:50.033956 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da710d4-b28f-4c94-94a4-e4307d551eca admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:50.147589 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-1da710d4-b28f-4c94-94a4-e4307d551eca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca23062d165c4b72ab4606722559e51e', 'request_id': 'req-1da710d4-b28f-4c94-94a4-e4307d551eca'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:50.153708 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-1da710d4-b28f-4c94-94a4-e4307d551eca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca23062d165c4b72ab4606722559e51e', 'request_id': 'req-1da710d4-b28f-4c94-94a4-e4307d551eca'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:50.233987 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-1da710d4-b28f-4c94-94a4-e4307d551eca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca23062d165c4b72ab4606722559e51e', 'request_id': 'req-1da710d4-b28f-4c94-94a4-e4307d551eca'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:50.249426 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-1da710d4-b28f-4c94-94a4-e4307d551eca admin admin] The token cache is being invalidate because project ca23062d165c4b72ab4606722559e51e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:09:50.249827 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-1da710d4-b28f-4c94-94a4-e4307d551eca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:50.250656 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-1da710d4-b28f-4c94-94a4-e4307d551eca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:50.253259 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 954/1944] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:09:49 2022] DELETE /identity/v3/projects/ca23062d165c4b72ab4606722559e51e => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:09:50.257145 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe79ddd-1e52-47b8-b049-0ec179cf29a4 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:50.257611 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe79ddd-1e52-47b8-b049-0ec179cf29a4 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:50.257713 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe79ddd-1e52-47b8-b049-0ec179cf29a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:50.283329 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-cbe79ddd-1e52-47b8-b049-0ec179cf29a4 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:09:50.284683 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-cbe79ddd-1e52-47b8-b049-0ec179cf29a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:09:50.352896 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 991/1945] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:09:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:09:50.361366 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8b941c-457f-4da6-97cd-d74aa33f220e None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:50.361661 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8b941c-457f-4da6-97cd-d74aa33f220e None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:50.361984 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8b941c-457f-4da6-97cd-d74aa33f220e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:50.382821 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-1b8b941c-457f-4da6-97cd-d74aa33f220e None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:09:50.384178 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-1b8b941c-457f-4da6-97cd-d74aa33f220e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:09:50.394096 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 955/1946] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:09:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:09:50.403246 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe79ddd-1e52-47b8-b049-0ec179cf29a4 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:50.412928 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe79ddd-1e52-47b8-b049-0ec179cf29a4 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:50.417487 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2942f3d-83fb-481f-b09d-3f1c0ed19e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:50.418113 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2942f3d-83fb-481f-b09d-3f1c0ed19e5b admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:50.418392 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2942f3d-83fb-481f-b09d-3f1c0ed19e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:50.418664 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2942f3d-83fb-481f-b09d-3f1c0ed19e5b admin admin] PATH_INFO: `/v3/domains` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:50.419859 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2942f3d-83fb-481f-b09d-3f1c0ed19e5b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 03:09:50.420161 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2942f3d-83fb-481f-b09d-3f1c0ed19e5b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:50.421750 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2942f3d-83fb-481f-b09d-3f1c0ed19e5b admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:50.423213 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:09:50.433315 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 992/1947] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Oct 14 03:09:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 03:09:50.438346 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8b941c-457f-4da6-97cd-d74aa33f220e None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:50.447601 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8b941c-457f-4da6-97cd-d74aa33f220e None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:50.452243 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea1f8bc-ae40-408c-9b36-4351ff9bf7a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:50.452868 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea1f8bc-ae40-408c-9b36-4351ff9bf7a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:50.453151 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea1f8bc-ae40-408c-9b36-4351ff9bf7a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:50.453423 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea1f8bc-ae40-408c-9b36-4351ff9bf7a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:50.454094 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea1f8bc-ae40-408c-9b36-4351ff9bf7a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:50.455985 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea1f8bc-ae40-408c-9b36-4351ff9bf7a9 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:50.607467 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 956/1948] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:09:50 2022] POST /identity/v3/projects => generated 373 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:09:50.612217 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2942f3d-83fb-481f-b09d-3f1c0ed19e5b admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:50.624070 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2942f3d-83fb-481f-b09d-3f1c0ed19e5b admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:50.700392 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2644f5fe-3861-4ee4-9a0b-7055fc0a4766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:50.701014 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2644f5fe-3861-4ee4-9a0b-7055fc0a4766 admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:50.701303 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2644f5fe-3861-4ee4-9a0b-7055fc0a4766 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:50.701582 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2644f5fe-3861-4ee4-9a0b-7055fc0a4766 admin admin] PATH_INFO: `/v3/users` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:50.702269 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2644f5fe-3861-4ee4-9a0b-7055fc0a4766 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:50.704405 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2644f5fe-3861-4ee4-9a0b-7055fc0a4766 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:50.811846 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 993/1949] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:09:50 2022] POST /identity/v3/users => generated 287 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:09:50.816294 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea1f8bc-ae40-408c-9b36-4351ff9bf7a9 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:50.831130 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea1f8bc-ae40-408c-9b36-4351ff9bf7a9 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:50.839585 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cea7adb-8888-4ae8-b7bb-580eeb495937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:50.840292 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cea7adb-8888-4ae8-b7bb-580eeb495937 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:50.841458 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cea7adb-8888-4ae8-b7bb-580eeb495937 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:50.841878 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cea7adb-8888-4ae8-b7bb-580eeb495937 admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:50.842634 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cea7adb-8888-4ae8-b7bb-580eeb495937 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:50.844680 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cea7adb-8888-4ae8-b7bb-580eeb495937 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:50.855977 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 957/1950] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:09:50 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:09:50.861125 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2644f5fe-3861-4ee4-9a0b-7055fc0a4766 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:50.877893 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2644f5fe-3861-4ee4-9a0b-7055fc0a4766 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:50.889593 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76d0549-610f-410f-ba0b-a6a7af46560c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:50.893228 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cea7adb-8888-4ae8-b7bb-580eeb495937 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:50.893356 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76d0549-610f-410f-ba0b-a6a7af46560c admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:50.893356 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76d0549-610f-410f-ba0b-a6a7af46560c admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:50.893356 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76d0549-610f-410f-ba0b-a6a7af46560c admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:50.893356 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76d0549-610f-410f-ba0b-a6a7af46560c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:50.893685 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-5cea7adb-8888-4ae8-b7bb-580eeb495937 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:09:50.893807 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76d0549-610f-410f-ba0b-a6a7af46560c admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:50.905954 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cea7adb-8888-4ae8-b7bb-580eeb495937 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:50.906587 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 994/1951] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:09:50 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:09:50.910528 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76d0549-610f-410f-ba0b-a6a7af46560c admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:50.910961 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91ffb59-1202-4528-ab64-5db3d0d3be83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:50.911617 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91ffb59-1202-4528-ab64-5db3d0d3be83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:50.911899 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91ffb59-1202-4528-ab64-5db3d0d3be83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:50.912177 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91ffb59-1202-4528-ab64-5db3d0d3be83 admin admin] PATH_INFO: `/v3/users/a547dffc3e0147b9a91ec0f40ca08ccb` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:50.913332 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91ffb59-1202-4528-ab64-5db3d0d3be83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a547dffc3e0147b9a91ec0f40ca08ccb)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:50.915308 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91ffb59-1202-4528-ab64-5db3d0d3be83 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:50.920906 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76d0549-610f-410f-ba0b-a6a7af46560c admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:50.925784 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec4c16c-db26-47ab-afd0-659b0deda7d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:50.926796 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec4c16c-db26-47ab-afd0-659b0deda7d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:50.926796 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec4c16c-db26-47ab-afd0-659b0deda7d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:50.927024 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec4c16c-db26-47ab-afd0-659b0deda7d7 admin admin] PATH_INFO: `/v3/projects/57c8543c66ed4a6889a7740932dc6433/users/e9b2d8f2742d4e5a8576e16c122c95ac/roles/9bd0470b544240209a420f640c79c06c` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:50.947484 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec4c16c-db26-47ab-afd0-659b0deda7d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57c8543c66ed4a6889a7740932dc6433, user_id=e9b2d8f2742d4e5a8576e16c122c95ac, role_id=9bd0470b544240209a420f640c79c06c)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:50.952574 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec4c16c-db26-47ab-afd0-659b0deda7d7 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:51.030699 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 995/1952] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:09:50 2022] PUT /identity/v3/projects/57c8543c66ed4a6889a7740932dc6433/users/e9b2d8f2742d4e5a8576e16c122c95ac/roles/9bd0470b544240209a420f640c79c06c => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 03:09:51.039159 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e9de7d-3324-4634-bc7a-70b32f690923 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:51.041136 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e9de7d-3324-4634-bc7a-70b32f690923 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:51.042914 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e9de7d-3324-4634-bc7a-70b32f690923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:51.054504 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-f91ffb59-1202-4528-ab64-5db3d0d3be83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a547dffc3e0147b9a91ec0f40ca08ccb'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:51.072268 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-f91ffb59-1202-4528-ab64-5db3d0d3be83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a547dffc3e0147b9a91ec0f40ca08ccb'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:51.075815 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:09:51.079341 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-51e9de7d-3324-4634-bc7a-70b32f690923 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:09:51.084122 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-51e9de7d-3324-4634-bc7a-70b32f690923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:09:51.089724 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-f91ffb59-1202-4528-ab64-5db3d0d3be83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a547dffc3e0147b9a91ec0f40ca08ccb'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:51.090528 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-f91ffb59-1202-4528-ab64-5db3d0d3be83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a547dffc3e0147b9a91ec0f40ca08ccb'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:51.148050 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 996/1953] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:09:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:09:51.164091 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e9de7d-3324-4634-bc7a-70b32f690923 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:51.165640 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-51e9de7d-3324-4634-bc7a-70b32f690923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:09:51.174549 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-f91ffb59-1202-4528-ab64-5db3d0d3be83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a547dffc3e0147b9a91ec0f40ca08ccb'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:51.181780 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 958/1954] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:09:50 2022] DELETE /identity/v3/users/a547dffc3e0147b9a91ec0f40ca08ccb => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:09:51.185915 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d16c00f-3ba0-4172-8ff1-120e74409981 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:51.186200 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d16c00f-3ba0-4172-8ff1-120e74409981 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:51.186697 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d16c00f-3ba0-4172-8ff1-120e74409981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:51.218010 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-4d16c00f-3ba0-4172-8ff1-120e74409981 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:09:51.219771 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-4d16c00f-3ba0-4172-8ff1-120e74409981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:09:51.244218 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e9de7d-3324-4634-bc7a-70b32f690923 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:51.268560 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:09:51.302206 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e664d83-04eb-4412-842b-67ad3cd268a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:51.303077 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e664d83-04eb-4412-842b-67ad3cd268a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:51.303645 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e664d83-04eb-4412-842b-67ad3cd268a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:51.304183 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e664d83-04eb-4412-842b-67ad3cd268a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:51.306456 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 959/1955] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:09:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 03:09:51.313510 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d16c00f-3ba0-4172-8ff1-120e74409981 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:51.322115 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e664d83-04eb-4412-842b-67ad3cd268a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:51.323570 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e664d83-04eb-4412-842b-67ad3cd268a5 service neutron] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:51.323702 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d16c00f-3ba0-4172-8ff1-120e74409981 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:51.328023 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c5afaf-6090-4887-a14f-016e9f91348e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:51.328646 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c5afaf-6090-4887-a14f-016e9f91348e service neutron] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:51.329103 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c5afaf-6090-4887-a14f-016e9f91348e service neutron] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:51.329379 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c5afaf-6090-4887-a14f-016e9f91348e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:51.335625 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 997/1956] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:09:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:09:51.340398 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c5afaf-6090-4887-a14f-016e9f91348e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:51.343097 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c5afaf-6090-4887-a14f-016e9f91348e service neutron] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:51.355120 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 960/1957] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:09:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 03:09:51.852544 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e664d83-04eb-4412-842b-67ad3cd268a5 service neutron] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:51.854445 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-6e664d83-04eb-4412-842b-67ad3cd268a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:09:51.872877 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e664d83-04eb-4412-842b-67ad3cd268a5 service neutron] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:51.880097 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cb37a6-7551-44e5-bb2b-51ed92095ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:51.880750 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb37a6-7551-44e5-bb2b-51ed92095ece admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:51.881092 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb37a6-7551-44e5-bb2b-51ed92095ece admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:51.881689 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb37a6-7551-44e5-bb2b-51ed92095ece admin admin] PATH_INFO: `/v3/projects/73cab3822d4d43c2a1c6ccac1e2d984c` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:51.883007 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cb37a6-7551-44e5-bb2b-51ed92095ece admin admin] RBAC: Authorizing `identity:delete_project(project_id=73cab3822d4d43c2a1c6ccac1e2d984c)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:51.885007 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cb37a6-7551-44e5-bb2b-51ed92095ece admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:52.070381 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-b1cb37a6-7551-44e5-bb2b-51ed92095ece admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73cab3822d4d43c2a1c6ccac1e2d984c', 'request_id': 'req-b1cb37a6-7551-44e5-bb2b-51ed92095ece'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:52.078044 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-b1cb37a6-7551-44e5-bb2b-51ed92095ece admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73cab3822d4d43c2a1c6ccac1e2d984c', 'request_id': 'req-b1cb37a6-7551-44e5-bb2b-51ed92095ece'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:52.142227 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-b1cb37a6-7551-44e5-bb2b-51ed92095ece admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73cab3822d4d43c2a1c6ccac1e2d984c', 'request_id': 'req-b1cb37a6-7551-44e5-bb2b-51ed92095ece'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:52.167803 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-b1cb37a6-7551-44e5-bb2b-51ed92095ece admin admin] The token cache is being invalidate because project 73cab3822d4d43c2a1c6ccac1e2d984c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81517) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:09:52.168132 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-b1cb37a6-7551-44e5-bb2b-51ed92095ece admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:52.169582 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-b1cb37a6-7551-44e5-bb2b-51ed92095ece admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:09:52.177273 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 998/1958] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:09:51 2022] DELETE /identity/v3/projects/73cab3822d4d43c2a1c6ccac1e2d984c => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:09:52.677818 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9feeed-5384-4739-8c10-f7638d14996e None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:52.678336 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9feeed-5384-4739-8c10-f7638d14996e None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:52.678419 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9feeed-5384-4739-8c10-f7638d14996e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:52.706792 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-5f9feeed-5384-4739-8c10-f7638d14996e None None] MFA Rules not processed for user `e9b2d8f2742d4e5a8576e16c122c95ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:09:52.708027 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-5f9feeed-5384-4739-8c10-f7638d14996e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:09:52.777803 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 961/1959] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:09:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:09:52.788826 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cb37a6-7551-44e5-bb2b-51ed92095ece admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:09:52.790295 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-b1cb37a6-7551-44e5-bb2b-51ed92095ece admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:09:52.847833 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cb37a6-7551-44e5-bb2b-51ed92095ece admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:09:52.851364 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f0a665-f805-4100-a972-c6debaf361f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab599d98b0624a2e89090312bcab713f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:09:52.851968 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f0a665-f805-4100-a972-c6debaf361f4 service cinder] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:52.852252 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f0a665-f805-4100-a972-c6debaf361f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:52.852529 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f0a665-f805-4100-a972-c6debaf361f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:52.860890 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f0a665-f805-4100-a972-c6debaf361f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:09:52.862375 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f0a665-f805-4100-a972-c6debaf361f4 service cinder] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:09:52.871032 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 999/1960] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:09:52 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:09:56.816980 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1115189-f4b3-44d3-86bb-0d52fb26247a None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:09:56.817511 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1115189-f4b3-44d3-86bb-0d52fb26247a None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:09:56.817511 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1115189-f4b3-44d3-86bb-0d52fb26247a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:09:56.837633 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-b1115189-f4b3-44d3-86bb-0d52fb26247a None None] MFA Rules not processed for user `e9b2d8f2742d4e5a8576e16c122c95ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:09:56.838872 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-b1115189-f4b3-44d3-86bb-0d52fb26247a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:09:56.846768 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 962/1961] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:09:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:10:02.867023 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=77,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:10:06.832057 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=76,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:10:11.152622 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77a5eab-b299-4fd5-9e4f-85902aa3844d None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:10:11.157131 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77a5eab-b299-4fd5-9e4f-85902aa3844d None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:10:11.157131 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77a5eab-b299-4fd5-9e4f-85902aa3844d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:10:11.225877 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-a77a5eab-b299-4fd5-9e4f-85902aa3844d None None] MFA Rules not processed for user `44b37050076b42e6af1a67bd7d6d06e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:10:11.232315 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-a77a5eab-b299-4fd5-9e4f-85902aa3844d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:10:11.318256 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1000/1962] 10.222.0.39 () {58 vars in 1009 bytes} [Fri Oct 14 03:10:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:10:11.367562 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1115189-f4b3-44d3-86bb-0d52fb26247a None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:10:11.368990 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-b1115189-f4b3-44d3-86bb-0d52fb26247a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:10:11.440586 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1115189-f4b3-44d3-86bb-0d52fb26247a None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:10:11.445521 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f938136-f933-46f0-be08-4ba1ca88c9a7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5e07da64804243bc18862072b027a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:10:11.446202 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f938136-f933-46f0-be08-4ba1ca88c9a7 service swift] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:10:11.446480 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f938136-f933-46f0-be08-4ba1ca88c9a7 service swift] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:10:11.446751 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f938136-f933-46f0-be08-4ba1ca88c9a7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:10:11.455811 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f938136-f933-46f0-be08-4ba1ca88c9a7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:10:11.457429 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f938136-f933-46f0-be08-4ba1ca88c9a7 service swift] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:10:11.468028 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 963/1963] 10.222.0.39 () {60 vars in 1384 bytes} [Fri Oct 14 03:10:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 14 03:10:21.310167 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=25 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:10:21.462619 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=32 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:10:33.221535 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a29ed1-8a78-4f65-bd71-595d4221af82 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:10:33.222119 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a29ed1-8a78-4f65-bd71-595d4221af82 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:10:33.222119 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a29ed1-8a78-4f65-bd71-595d4221af82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:10:33.241426 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-81a29ed1-8a78-4f65-bd71-595d4221af82 None None] MFA Rules not processed for user `ced832cf86304814b020de4ed3e0955f`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:10:33.242959 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-81a29ed1-8a78-4f65-bd71-595d4221af82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:10:33.299460 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1001/1964] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:10:33 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:10:35.091385 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5279d4c7-2ee1-4067-b142-0971b6662ec9 None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:10:35.099265 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5279d4c7-2ee1-4067-b142-0971b6662ec9 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:10:35.099265 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5279d4c7-2ee1-4067-b142-0971b6662ec9 None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:10:35.099265 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 964/1965] 10.222.0.39 () {54 vars in 921 bytes} [Fri Oct 14 03:10:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 03:10:35.101104 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ed25ee-6f91-4dbf-85ad-42e429c90e42 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:10:35.101597 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ed25ee-6f91-4dbf-85ad-42e429c90e42 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:10:35.101690 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ed25ee-6f91-4dbf-85ad-42e429c90e42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:10:35.122446 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-96ed25ee-6f91-4dbf-85ad-42e429c90e42 None None] MFA Rules not processed for user `a6b07bf0faca47b28b50990d1720853f`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:10:35.123344 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba3698e-a755-49cd-9402-8042689aa4f7 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:10:35.123633 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba3698e-a755-49cd-9402-8042689aa4f7 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:10:35.123908 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba3698e-a755-49cd-9402-8042689aa4f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:10:35.124413 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-96ed25ee-6f91-4dbf-85ad-42e429c90e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:10:35.146846 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-cba3698e-a755-49cd-9402-8042689aa4f7 None None] MFA Rules not processed for user `e9b2d8f2742d4e5a8576e16c122c95ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:10:35.148154 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-cba3698e-a755-49cd-9402-8042689aa4f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:10:35.155305 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 965/1966] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:10:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 03:10:35.201217 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1002/1967] 10.222.0.39 () {58 vars in 1018 bytes} [Fri Oct 14 03:10:35 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:10:35.217429 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba3698e-a755-49cd-9402-8042689aa4f7 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:10:35.219846 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-cba3698e-a755-49cd-9402-8042689aa4f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:10:35.313381 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba3698e-a755-49cd-9402-8042689aa4f7 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:10:35.318795 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef565b76-b8eb-48fb-81eb-cba6db7407a4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a3cd1fbe6df48f8aa90d4ea69905e9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:10:35.319412 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef565b76-b8eb-48fb-81eb-cba6db7407a4 service glance] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:10:35.319694 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef565b76-b8eb-48fb-81eb-cba6db7407a4 service glance] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:10:35.319997 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef565b76-b8eb-48fb-81eb-cba6db7407a4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:10:35.331633 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef565b76-b8eb-48fb-81eb-cba6db7407a4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:10:35.333132 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef565b76-b8eb-48fb-81eb-cba6db7407a4 service glance] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:10:35.344966 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 966/1968] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:10:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:10:35.586972 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b0c1db-ffae-4ba0-bdd4-cd6246e593a8 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:10:35.587257 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b0c1db-ffae-4ba0-bdd4-cd6246e593a8 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:10:35.587531 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b0c1db-ffae-4ba0-bdd4-cd6246e593a8 None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:10:35.588912 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1003/1969] 10.222.0.39 () {54 vars in 921 bytes} [Fri Oct 14 03:10:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 03:10:35.594183 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c9e6c3-6974-4c89-8ce0-72d34925bf9a None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:10:35.594471 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c9e6c3-6974-4c89-8ce0-72d34925bf9a None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:10:35.595081 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c9e6c3-6974-4c89-8ce0-72d34925bf9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:10:35.615167 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-b7c9e6c3-6974-4c89-8ce0-72d34925bf9a None None] MFA Rules not processed for user `b123c412ac714b06929a4bd86eca71e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:10:35.616419 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-b7c9e6c3-6974-4c89-8ce0-72d34925bf9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:10:35.670633 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 967/1970] 10.222.0.39 () {58 vars in 1018 bytes} [Fri Oct 14 03:10:35 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:10:35.683394 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b0c1db-ffae-4ba0-bdd4-cd6246e593a8 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:10:35.685118 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-20b0c1db-ffae-4ba0-bdd4-cd6246e593a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:10:35.699037 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b0c1db-ffae-4ba0-bdd4-cd6246e593a8 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:10:35.703105 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d4899b-4217-4085-948d-182bbd9f82b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:10:35.703713 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d4899b-4217-4085-948d-182bbd9f82b9 service neutron] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:10:35.703990 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d4899b-4217-4085-948d-182bbd9f82b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:10:35.704257 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d4899b-4217-4085-948d-182bbd9f82b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:10:35.711858 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d4899b-4217-4085-948d-182bbd9f82b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:10:35.713320 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d4899b-4217-4085-948d-182bbd9f82b9 service neutron] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:10:35.721302 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1004/1971] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:10:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 03:10:35.808153 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99778dfd-5af7-4038-ab90-778721cab21e None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:10:35.808528 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99778dfd-5af7-4038-ab90-778721cab21e None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:10:35.808878 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99778dfd-5af7-4038-ab90-778721cab21e None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:10:35.811008 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 968/1972] 10.222.0.39 () {54 vars in 921 bytes} [Fri Oct 14 03:10:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 03:10:35.817213 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28823f6-2ef5-462f-a96d-adbed0770bf5 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:10:35.817506 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28823f6-2ef5-462f-a96d-adbed0770bf5 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:10:35.818010 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28823f6-2ef5-462f-a96d-adbed0770bf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:10:35.837721 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-a28823f6-2ef5-462f-a96d-adbed0770bf5 None None] MFA Rules not processed for user `a6b07bf0faca47b28b50990d1720853f`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:10:35.838950 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-a28823f6-2ef5-462f-a96d-adbed0770bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:10:35.849082 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1005/1973] 10.222.0.39 () {58 vars in 1018 bytes} [Fri Oct 14 03:10:35 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:10:35.860409 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99778dfd-5af7-4038-ab90-778721cab21e None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:10:35.873208 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99778dfd-5af7-4038-ab90-778721cab21e None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:10:35.879392 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7e3289-c4c2-4342-a640-f3d3ca9a4d54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:10:35.880505 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7e3289-c4c2-4342-a640-f3d3ca9a4d54 service neutron] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:10:35.880978 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7e3289-c4c2-4342-a640-f3d3ca9a4d54 service neutron] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:10:35.881525 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7e3289-c4c2-4342-a640-f3d3ca9a4d54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:10:35.893400 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7e3289-c4c2-4342-a640-f3d3ca9a4d54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:10:35.894954 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7e3289-c4c2-4342-a640-f3d3ca9a4d54 service neutron] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:10:35.906249 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 969/1974] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:10:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:10:38.441953 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28823f6-2ef5-462f-a96d-adbed0770bf5 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:10:38.443453 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-a28823f6-2ef5-462f-a96d-adbed0770bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:10:38.535819 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28823f6-2ef5-462f-a96d-adbed0770bf5 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:10:38.541977 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9f8c33-8593-4e39-a5aa-9f2b362cdad9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '922ef7811aed450baff7f936011b0af1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:10:38.543260 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f8c33-8593-4e39-a5aa-9f2b362cdad9 service placement] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:10:38.543260 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f8c33-8593-4e39-a5aa-9f2b362cdad9 service placement] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:10:38.543513 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f8c33-8593-4e39-a5aa-9f2b362cdad9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:10:38.545157 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-3e9f8c33-8593-4e39-a5aa-9f2b362cdad9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:10:38.561802 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9f8c33-8593-4e39-a5aa-9f2b362cdad9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:10:38.562457 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9f8c33-8593-4e39-a5aa-9f2b362cdad9 service placement] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:10:38.575580 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1006/1975] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:10:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:10:42.382702 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ea0f0e-f4b8-4498-aa9e-dc28faa6c592 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:10:42.383216 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ea0f0e-f4b8-4498-aa9e-dc28faa6c592 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:10:42.383415 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ea0f0e-f4b8-4498-aa9e-dc28faa6c592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:10:42.403335 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-d0ea0f0e-f4b8-4498-aa9e-dc28faa6c592 None None] MFA Rules not processed for user `44b37050076b42e6af1a67bd7d6d06e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:10:42.404440 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-d0ea0f0e-f4b8-4498-aa9e-dc28faa6c592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:10:42.412559 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 970/1976] 10.222.0.39 () {58 vars in 1009 bytes} [Fri Oct 14 03:10:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:10:42.425697 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9f8c33-8593-4e39-a5aa-9f2b362cdad9 service placement] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:10:42.437440 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9f8c33-8593-4e39-a5aa-9f2b362cdad9 service placement] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:10:42.442528 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5730cc04-305c-4254-8c4d-11425d988bb7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5e07da64804243bc18862072b027a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:10:42.443141 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5730cc04-305c-4254-8c4d-11425d988bb7 service swift] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:10:42.444011 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5730cc04-305c-4254-8c4d-11425d988bb7 service swift] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:10:42.444793 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5730cc04-305c-4254-8c4d-11425d988bb7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:10:42.449964 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:10:42.454391 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5730cc04-305c-4254-8c4d-11425d988bb7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:10:42.456066 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5730cc04-305c-4254-8c4d-11425d988bb7 service swift] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:10:42.470000 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1007/1977] 10.222.0.39 () {60 vars in 1384 bytes} [Fri Oct 14 03:10:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 14 03:10:52.398078 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=70 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:10:52.542627 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=3 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:11:08.853300 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb44e5b7-16a6-4988-a46c-f75d73179086 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:11:08.853835 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb44e5b7-16a6-4988-a46c-f75d73179086 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:11:08.853929 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb44e5b7-16a6-4988-a46c-f75d73179086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:11:08.875651 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-eb44e5b7-16a6-4988-a46c-f75d73179086 None None] MFA Rules not processed for user `44b37050076b42e6af1a67bd7d6d06e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:11:08.878295 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-eb44e5b7-16a6-4988-a46c-f75d73179086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:11:08.885646 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 971/1978] 10.222.0.39 () {58 vars in 967 bytes} [Fri Oct 14 03:11:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:11:08.899338 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5730cc04-305c-4254-8c4d-11425d988bb7 service swift] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:11:08.910826 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5730cc04-305c-4254-8c4d-11425d988bb7 service swift] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:11:08.915895 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0625694-45a3-4695-9dc6-fd83e2c57e6f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5e07da64804243bc18862072b027a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:11:08.916498 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0625694-45a3-4695-9dc6-fd83e2c57e6f service swift] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:11:08.916783 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0625694-45a3-4695-9dc6-fd83e2c57e6f service swift] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:11:08.917061 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0625694-45a3-4695-9dc6-fd83e2c57e6f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:11:08.925678 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0625694-45a3-4695-9dc6-fd83e2c57e6f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:11:08.931465 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0625694-45a3-4695-9dc6-fd83e2c57e6f service swift] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:11:08.935834 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1008/1979] 10.222.0.39 () {60 vars in 1384 bytes} [Fri Oct 14 03:11:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 14 03:11:18.868998 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=3 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:11:18.932932 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=6 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:11:24.459387 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6827f67d-8564-4f46-b90d-127c849458d8 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:11:24.459883 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6827f67d-8564-4f46-b90d-127c849458d8 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:11:24.459988 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6827f67d-8564-4f46-b90d-127c849458d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:11:24.479394 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-6827f67d-8564-4f46-b90d-127c849458d8 None None] MFA Rules not processed for user `44b37050076b42e6af1a67bd7d6d06e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:11:24.480651 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-6827f67d-8564-4f46-b90d-127c849458d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:11:24.488534 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 972/1980] 10.222.0.39 () {58 vars in 1009 bytes} [Fri Oct 14 03:11:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:11:24.511472 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0625694-45a3-4695-9dc6-fd83e2c57e6f service swift] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:11:24.523304 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0625694-45a3-4695-9dc6-fd83e2c57e6f service swift] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:11:24.528620 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a480f24-9bed-4b4c-8408-5a8022f4cbfd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5e07da64804243bc18862072b027a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:11:24.529228 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a480f24-9bed-4b4c-8408-5a8022f4cbfd service swift] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:11:24.529628 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a480f24-9bed-4b4c-8408-5a8022f4cbfd service swift] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:11:24.529951 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a480f24-9bed-4b4c-8408-5a8022f4cbfd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:11:24.539137 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a480f24-9bed-4b4c-8408-5a8022f4cbfd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:11:24.540777 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a480f24-9bed-4b4c-8408-5a8022f4cbfd service swift] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:11:24.550019 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1009/1981] 10.222.0.39 () {60 vars in 1384 bytes} [Fri Oct 14 03:11:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 14 03:11:30.334380 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5925662-91d5-4137-b3e8-7a942daff2c6 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:11:30.334991 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5925662-91d5-4137-b3e8-7a942daff2c6 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:11:30.335109 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5925662-91d5-4137-b3e8-7a942daff2c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:11:30.361994 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-d5925662-91d5-4137-b3e8-7a942daff2c6 None None] MFA Rules not processed for user `410b7c9e93c94ce78d1b5c0e646307a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:11:30.361994 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-d5925662-91d5-4137-b3e8-7a942daff2c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:11:30.434450 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 973/1982] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:11:30 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:11:34.546802 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=6 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:11:40.422914 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=28 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:11:46.632453 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f1fd55-ea19-49f4-8fb6-619860c75da2 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:11:46.632914 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f1fd55-ea19-49f4-8fb6-619860c75da2 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:11:46.633004 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f1fd55-ea19-49f4-8fb6-619860c75da2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:11:46.652588 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-d4f1fd55-ea19-49f4-8fb6-619860c75da2 None None] MFA Rules not processed for user `44b37050076b42e6af1a67bd7d6d06e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:11:46.653848 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-d4f1fd55-ea19-49f4-8fb6-619860c75da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:11:46.660975 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1010/1983] 10.222.0.39 () {58 vars in 967 bytes} [Fri Oct 14 03:11:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:11:46.673568 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5925662-91d5-4137-b3e8-7a942daff2c6 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:11:46.685808 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5925662-91d5-4137-b3e8-7a942daff2c6 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:11:46.691342 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca14756a-7d45-4c5a-81c9-05f67c8a0ce5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5e07da64804243bc18862072b027a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:11:46.691954 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca14756a-7d45-4c5a-81c9-05f67c8a0ce5 service swift] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:11:46.692240 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca14756a-7d45-4c5a-81c9-05f67c8a0ce5 service swift] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:11:46.692518 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca14756a-7d45-4c5a-81c9-05f67c8a0ce5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:11:46.700552 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca14756a-7d45-4c5a-81c9-05f67c8a0ce5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:11:46.702724 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca14756a-7d45-4c5a-81c9-05f67c8a0ce5 service swift] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:11:46.710739 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 974/1984] 10.222.0.39 () {60 vars in 1384 bytes} [Fri Oct 14 03:11:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 14 03:11:49.759428 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f1fd55-ea19-49f4-8fb6-619860c75da2 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:11:49.760927 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-d4f1fd55-ea19-49f4-8fb6-619860c75da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:11:49.847446 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f1fd55-ea19-49f4-8fb6-619860c75da2 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:11:49.853475 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a86991c-19c4-4592-8101-9329d9223b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:11:49.854182 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a86991c-19c4-4592-8101-9329d9223b99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:11:49.854462 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a86991c-19c4-4592-8101-9329d9223b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:11:49.854737 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a86991c-19c4-4592-8101-9329d9223b99 admin admin] PATH_INFO: `/v3/users/e9b2d8f2742d4e5a8576e16c122c95ac` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:11:49.855883 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a86991c-19c4-4592-8101-9329d9223b99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9b2d8f2742d4e5a8576e16c122c95ac)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:11:49.857897 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a86991c-19c4-4592-8101-9329d9223b99 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:11:50.083084 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-1a86991c-19c4-4592-8101-9329d9223b99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9b2d8f2742d4e5a8576e16c122c95ac'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:11:50.083743 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-1a86991c-19c4-4592-8101-9329d9223b99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9b2d8f2742d4e5a8576e16c122c95ac'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:11:50.126207 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-1a86991c-19c4-4592-8101-9329d9223b99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9b2d8f2742d4e5a8576e16c122c95ac'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:11:50.126787 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-1a86991c-19c4-4592-8101-9329d9223b99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9b2d8f2742d4e5a8576e16c122c95ac'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:11:50.136889 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-1a86991c-19c4-4592-8101-9329d9223b99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9b2d8f2742d4e5a8576e16c122c95ac'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:11:50.162244 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1011/1985] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:11:49 2022] DELETE /identity/v3/users/e9b2d8f2742d4e5a8576e16c122c95ac => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:11:50.681921 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca14756a-7d45-4c5a-81c9-05f67c8a0ce5 service swift] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:11:50.693058 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca14756a-7d45-4c5a-81c9-05f67c8a0ce5 service swift] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:11:50.751332 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7723ced-cf13-4b7f-b6ee-92657b58481a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:11:50.751953 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7723ced-cf13-4b7f-b6ee-92657b58481a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:11:50.752232 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7723ced-cf13-4b7f-b6ee-92657b58481a admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:11:50.752506 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7723ced-cf13-4b7f-b6ee-92657b58481a admin admin] PATH_INFO: `/v3/projects/57c8543c66ed4a6889a7740932dc6433` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:11:50.753651 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7723ced-cf13-4b7f-b6ee-92657b58481a admin admin] RBAC: Authorizing `identity:delete_project(project_id=57c8543c66ed4a6889a7740932dc6433)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:11:50.755572 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7723ced-cf13-4b7f-b6ee-92657b58481a admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:11:50.880316 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-a7723ced-cf13-4b7f-b6ee-92657b58481a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57c8543c66ed4a6889a7740932dc6433', 'request_id': 'req-a7723ced-cf13-4b7f-b6ee-92657b58481a'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:11:50.886530 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-a7723ced-cf13-4b7f-b6ee-92657b58481a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57c8543c66ed4a6889a7740932dc6433', 'request_id': 'req-a7723ced-cf13-4b7f-b6ee-92657b58481a'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:11:50.977595 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-a7723ced-cf13-4b7f-b6ee-92657b58481a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57c8543c66ed4a6889a7740932dc6433', 'request_id': 'req-a7723ced-cf13-4b7f-b6ee-92657b58481a'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:11:50.988588 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-a7723ced-cf13-4b7f-b6ee-92657b58481a admin admin] The token cache is being invalidate because project 57c8543c66ed4a6889a7740932dc6433 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:11:50.988872 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-a7723ced-cf13-4b7f-b6ee-92657b58481a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:11:50.989491 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-a7723ced-cf13-4b7f-b6ee-92657b58481a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:11:50.990975 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 975/1986] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:11:50 2022] DELETE /identity/v3/projects/57c8543c66ed4a6889a7740932dc6433 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:11:50.995835 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a4520c-da9d-43dd-853f-1514043585f3 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:11:50.996335 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a4520c-da9d-43dd-853f-1514043585f3 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:11:50.996441 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a4520c-da9d-43dd-853f-1514043585f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:11:51.018638 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-c1a4520c-da9d-43dd-853f-1514043585f3 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:11:51.019871 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-c1a4520c-da9d-43dd-853f-1514043585f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:11:51.084430 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1012/1987] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:11:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:11:51.092286 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8655046b-b5dc-4fa6-bc89-44080240d75e None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:11:51.093101 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8655046b-b5dc-4fa6-bc89-44080240d75e None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:11:51.093381 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8655046b-b5dc-4fa6-bc89-44080240d75e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:11:51.128493 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-8655046b-b5dc-4fa6-bc89-44080240d75e None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:11:51.129769 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-8655046b-b5dc-4fa6-bc89-44080240d75e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:11:51.139731 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 976/1988] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:11:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:11:51.143653 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a4520c-da9d-43dd-853f-1514043585f3 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:11:51.153735 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a4520c-da9d-43dd-853f-1514043585f3 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:11:51.158299 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638e9ba4-0a30-4aa9-b11f-e84392848c5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:11:51.158903 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638e9ba4-0a30-4aa9-b11f-e84392848c5f admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:11:51.159219 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638e9ba4-0a30-4aa9-b11f-e84392848c5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:11:51.159500 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638e9ba4-0a30-4aa9-b11f-e84392848c5f admin admin] PATH_INFO: `/v3/domains` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:11:51.160136 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638e9ba4-0a30-4aa9-b11f-e84392848c5f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 03:11:51.160883 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638e9ba4-0a30-4aa9-b11f-e84392848c5f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:11:51.162727 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638e9ba4-0a30-4aa9-b11f-e84392848c5f admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:11:51.171822 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:11:51.175273 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1013/1989] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Oct 14 03:11:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 03:11:51.179637 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8655046b-b5dc-4fa6-bc89-44080240d75e None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:11:51.190213 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8655046b-b5dc-4fa6-bc89-44080240d75e None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:11:51.195264 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961d0428-9f30-442b-aa3c-27cbf2ad6216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:11:51.196373 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d0428-9f30-442b-aa3c-27cbf2ad6216 admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:11:51.196373 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d0428-9f30-442b-aa3c-27cbf2ad6216 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:11:51.196576 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d0428-9f30-442b-aa3c-27cbf2ad6216 admin admin] PATH_INFO: `/v3/projects` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:11:51.197387 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961d0428-9f30-442b-aa3c-27cbf2ad6216 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:11:51.199456 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961d0428-9f30-442b-aa3c-27cbf2ad6216 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:11:51.301133 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 977/1990] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:11:51 2022] POST /identity/v3/projects => generated 371 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 03:11:51.305114 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638e9ba4-0a30-4aa9-b11f-e84392848c5f admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:11:51.317103 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638e9ba4-0a30-4aa9-b11f-e84392848c5f admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:11:51.381472 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038a6578-70e3-403d-9d88-cd3037e8ce35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:11:51.381725 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038a6578-70e3-403d-9d88-cd3037e8ce35 admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:11:51.382037 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038a6578-70e3-403d-9d88-cd3037e8ce35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:11:51.382316 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038a6578-70e3-403d-9d88-cd3037e8ce35 admin admin] PATH_INFO: `/v3/users` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:11:51.382951 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038a6578-70e3-403d-9d88-cd3037e8ce35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:11:51.384921 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038a6578-70e3-403d-9d88-cd3037e8ce35 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:11:51.479551 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1014/1991] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:11:51 2022] POST /identity/v3/users => generated 286 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 03:11:51.484614 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961d0428-9f30-442b-aa3c-27cbf2ad6216 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:11:51.495196 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961d0428-9f30-442b-aa3c-27cbf2ad6216 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:11:51.500448 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e665ee4-0fb9-439c-b816-f22bab4cd464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:11:51.501056 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e665ee4-0fb9-439c-b816-f22bab4cd464 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:11:51.501338 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e665ee4-0fb9-439c-b816-f22bab4cd464 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:11:51.501639 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e665ee4-0fb9-439c-b816-f22bab4cd464 admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:11:51.502429 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e665ee4-0fb9-439c-b816-f22bab4cd464 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:11:51.504037 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e665ee4-0fb9-439c-b816-f22bab4cd464 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:11:51.515333 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 978/1992] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:11:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:11:51.518631 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038a6578-70e3-403d-9d88-cd3037e8ce35 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:11:51.529972 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038a6578-70e3-403d-9d88-cd3037e8ce35 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:11:51.534966 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d602e853-ff1d-4fe3-a00c-3d2f63ca5f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:11:51.535819 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d602e853-ff1d-4fe3-a00c-3d2f63ca5f70 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:11:51.536242 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d602e853-ff1d-4fe3-a00c-3d2f63ca5f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:11:51.536527 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d602e853-ff1d-4fe3-a00c-3d2f63ca5f70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:11:51.537176 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d602e853-ff1d-4fe3-a00c-3d2f63ca5f70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:11:51.538996 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d602e853-ff1d-4fe3-a00c-3d2f63ca5f70 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:11:51.550829 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1015/1993] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:11:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 03:11:51.560864 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e665ee4-0fb9-439c-b816-f22bab4cd464 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:11:51.571705 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e665ee4-0fb9-439c-b816-f22bab4cd464 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:11:51.576364 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d910b86e-34cf-41d5-9ab4-607b50b81f1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:11:51.581029 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d910b86e-34cf-41d5-9ab4-607b50b81f1e admin admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:11:51.581029 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d910b86e-34cf-41d5-9ab4-607b50b81f1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:11:51.581029 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d910b86e-34cf-41d5-9ab4-607b50b81f1e admin admin] PATH_INFO: `/v3/projects/d70fa5273d764ba48f5d9e8f2aee22f2/users/e94fdbe76b58434581801fba32268b5a/roles/9bd0470b544240209a420f640c79c06c` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:11:51.592868 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d910b86e-34cf-41d5-9ab4-607b50b81f1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d70fa5273d764ba48f5d9e8f2aee22f2, user_id=e94fdbe76b58434581801fba32268b5a, role_id=9bd0470b544240209a420f640c79c06c)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:11:51.595561 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d910b86e-34cf-41d5-9ab4-607b50b81f1e admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:11:51.709558 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 979/1994] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:11:51 2022] PUT /identity/v3/projects/d70fa5273d764ba48f5d9e8f2aee22f2/users/e94fdbe76b58434581801fba32268b5a/roles/9bd0470b544240209a420f640c79c06c => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:11:51.720193 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa998de-6d13-4311-95dd-2794516f7a87 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:11:51.720637 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa998de-6d13-4311-95dd-2794516f7a87 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:11:51.721681 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa998de-6d13-4311-95dd-2794516f7a87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:11:51.755644 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-0fa998de-6d13-4311-95dd-2794516f7a87 None None] MFA Rules not processed for user `e94fdbe76b58434581801fba32268b5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:11:51.756942 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-0fa998de-6d13-4311-95dd-2794516f7a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:11:51.833070 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1016/1995] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:11:51 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:11:51.844907 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d910b86e-34cf-41d5-9ab4-607b50b81f1e admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:11:51.846763 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-d910b86e-34cf-41d5-9ab4-607b50b81f1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:11:51.862229 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:11:51.904360 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d910b86e-34cf-41d5-9ab4-607b50b81f1e admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:11:51.908603 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da84ffe-af32-45a2-bce8-dbccaf5c7314 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab599d98b0624a2e89090312bcab713f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:11:51.909203 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da84ffe-af32-45a2-bce8-dbccaf5c7314 service cinder] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:11:51.909482 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da84ffe-af32-45a2-bce8-dbccaf5c7314 service cinder] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:11:51.909773 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da84ffe-af32-45a2-bce8-dbccaf5c7314 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:11:51.917821 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da84ffe-af32-45a2-bce8-dbccaf5c7314 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:11:51.919960 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da84ffe-af32-45a2-bce8-dbccaf5c7314 service cinder] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:11:51.928930 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 980/1996] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:11:51 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:11:51.953190 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa998de-6d13-4311-95dd-2794516f7a87 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:11:52.022984 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa998de-6d13-4311-95dd-2794516f7a87 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:11:52.027999 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8ea8ea-694e-4bba-b093-a97a895267df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:11:52.028636 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8ea8ea-694e-4bba-b093-a97a895267df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:11:52.028915 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8ea8ea-694e-4bba-b093-a97a895267df admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:11:52.029188 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8ea8ea-694e-4bba-b093-a97a895267df admin admin] PATH_INFO: `/v3/users/e94fdbe76b58434581801fba32268b5a` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:11:52.030843 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8ea8ea-694e-4bba-b093-a97a895267df admin admin] RBAC: Authorizing `identity:delete_user(user_id=e94fdbe76b58434581801fba32268b5a)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:11:52.032831 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8ea8ea-694e-4bba-b093-a97a895267df admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:11:52.044124 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:11:52.220431 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-8b8ea8ea-694e-4bba-b093-a97a895267df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e94fdbe76b58434581801fba32268b5a'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:11:52.221068 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-8b8ea8ea-694e-4bba-b093-a97a895267df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e94fdbe76b58434581801fba32268b5a'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:11:52.290134 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-8b8ea8ea-694e-4bba-b093-a97a895267df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e94fdbe76b58434581801fba32268b5a'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:11:52.290968 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-8b8ea8ea-694e-4bba-b093-a97a895267df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e94fdbe76b58434581801fba32268b5a'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:11:52.299104 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-8b8ea8ea-694e-4bba-b093-a97a895267df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e94fdbe76b58434581801fba32268b5a'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:11:52.322313 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1017/1997] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:11:51 2022] DELETE /identity/v3/users/e94fdbe76b58434581801fba32268b5a => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:11:52.327879 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28935b09-bb0a-4021-9cd4-f5aa7ec20417 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:11:52.328621 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28935b09-bb0a-4021-9cd4-f5aa7ec20417 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:11:52.329217 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28935b09-bb0a-4021-9cd4-f5aa7ec20417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:11:52.374986 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-28935b09-bb0a-4021-9cd4-f5aa7ec20417 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:11:52.376292 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-28935b09-bb0a-4021-9cd4-f5aa7ec20417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:11:52.448083 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 981/1998] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:11:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:11:52.458182 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8ea8ea-694e-4bba-b093-a97a895267df admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:11:52.463874 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-8b8ea8ea-694e-4bba-b093-a97a895267df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:11:52.519470 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8ea8ea-694e-4bba-b093-a97a895267df admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:11:52.523084 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d780a762-573c-4a15-89bf-1eee2b991e79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:11:52.523685 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d780a762-573c-4a15-89bf-1eee2b991e79 service neutron] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:11:52.523961 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d780a762-573c-4a15-89bf-1eee2b991e79 service neutron] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:11:52.524228 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d780a762-573c-4a15-89bf-1eee2b991e79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:11:52.534196 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d780a762-573c-4a15-89bf-1eee2b991e79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:11:52.535630 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d780a762-573c-4a15-89bf-1eee2b991e79 service neutron] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:11:52.545324 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1018/1999] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:11:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:11:52.957151 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28935b09-bb0a-4021-9cd4-f5aa7ec20417 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:11:52.958853 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-28935b09-bb0a-4021-9cd4-f5aa7ec20417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:11:52.972085 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28935b09-bb0a-4021-9cd4-f5aa7ec20417 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:11:52.976960 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801eeef2-901a-4a7f-9cfa-a641cb9a442c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:11:52.977599 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801eeef2-901a-4a7f-9cfa-a641cb9a442c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:11:52.977915 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801eeef2-901a-4a7f-9cfa-a641cb9a442c admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:11:52.978194 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801eeef2-901a-4a7f-9cfa-a641cb9a442c admin admin] PATH_INFO: `/v3/projects/d70fa5273d764ba48f5d9e8f2aee22f2` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:11:52.979320 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801eeef2-901a-4a7f-9cfa-a641cb9a442c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d70fa5273d764ba48f5d9e8f2aee22f2)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:11:52.981233 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801eeef2-901a-4a7f-9cfa-a641cb9a442c admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:11:53.139124 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-801eeef2-901a-4a7f-9cfa-a641cb9a442c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd70fa5273d764ba48f5d9e8f2aee22f2', 'request_id': 'req-801eeef2-901a-4a7f-9cfa-a641cb9a442c'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:11:53.145736 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-801eeef2-901a-4a7f-9cfa-a641cb9a442c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd70fa5273d764ba48f5d9e8f2aee22f2', 'request_id': 'req-801eeef2-901a-4a7f-9cfa-a641cb9a442c'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:11:53.212113 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-801eeef2-901a-4a7f-9cfa-a641cb9a442c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd70fa5273d764ba48f5d9e8f2aee22f2', 'request_id': 'req-801eeef2-901a-4a7f-9cfa-a641cb9a442c'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:11:53.224104 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-801eeef2-901a-4a7f-9cfa-a641cb9a442c admin admin] The token cache is being invalidate because project d70fa5273d764ba48f5d9e8f2aee22f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:11:53.224424 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-801eeef2-901a-4a7f-9cfa-a641cb9a442c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:11:53.225081 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-801eeef2-901a-4a7f-9cfa-a641cb9a442c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:11:53.226623 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 982/2000] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:11:52 2022] DELETE /identity/v3/projects/d70fa5273d764ba48f5d9e8f2aee22f2 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:11:53.231406 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe10884e-4640-458c-8206-9b89ef2ee5df None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:11:53.231838 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe10884e-4640-458c-8206-9b89ef2ee5df None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:11:53.231939 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe10884e-4640-458c-8206-9b89ef2ee5df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:11:53.253439 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-fe10884e-4640-458c-8206-9b89ef2ee5df None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:11:53.254672 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-fe10884e-4640-458c-8206-9b89ef2ee5df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:11:53.316398 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1019/2001] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:11:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 03:11:53.325611 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b0d06e-6c68-46cc-925f-6ed40f71ab5a None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:11:53.325933 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b0d06e-6c68-46cc-925f-6ed40f71ab5a None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:11:53.326209 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b0d06e-6c68-46cc-925f-6ed40f71ab5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:11:53.347243 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-17b0d06e-6c68-46cc-925f-6ed40f71ab5a None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:11:53.348440 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-17b0d06e-6c68-46cc-925f-6ed40f71ab5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:11:53.356564 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 983/2002] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:11:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:11:53.361671 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe10884e-4640-458c-8206-9b89ef2ee5df None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:11:53.371228 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe10884e-4640-458c-8206-9b89ef2ee5df None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:11:53.376655 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c7f2a8-b958-4ee0-bbb4-84fed82f026d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:11:53.377255 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c7f2a8-b958-4ee0-bbb4-84fed82f026d admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:11:53.377533 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c7f2a8-b958-4ee0-bbb4-84fed82f026d admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:11:53.377838 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c7f2a8-b958-4ee0-bbb4-84fed82f026d admin admin] PATH_INFO: `/v3/domains` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:11:53.378509 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c7f2a8-b958-4ee0-bbb4-84fed82f026d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 03:11:53.378811 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c7f2a8-b958-4ee0-bbb4-84fed82f026d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:11:53.380420 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c7f2a8-b958-4ee0-bbb4-84fed82f026d admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:11:53.395623 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1020/2003] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Oct 14 03:11:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 03:11:53.396839 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b0d06e-6c68-46cc-925f-6ed40f71ab5a None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:11:53.406653 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b0d06e-6c68-46cc-925f-6ed40f71ab5a None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:11:53.411306 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce586c17-48a5-498e-8330-265fa9e8b48a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:11:53.411907 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce586c17-48a5-498e-8330-265fa9e8b48a admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:11:53.412182 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce586c17-48a5-498e-8330-265fa9e8b48a admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:11:53.412453 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce586c17-48a5-498e-8330-265fa9e8b48a admin admin] PATH_INFO: `/v3/projects` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:11:53.413087 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce586c17-48a5-498e-8330-265fa9e8b48a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:11:53.414999 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce586c17-48a5-498e-8330-265fa9e8b48a admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:11:53.545982 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 984/2004] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:11:53 2022] POST /identity/v3/projects => generated 373 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:11:53.549959 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c7f2a8-b958-4ee0-bbb4-84fed82f026d admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:11:53.560506 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c7f2a8-b958-4ee0-bbb4-84fed82f026d admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:11:53.574847 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:11:53.621864 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d579f1f-34ec-4f82-8886-96c49d552e9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:11:53.622478 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d579f1f-34ec-4f82-8886-96c49d552e9c admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:11:53.622758 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d579f1f-34ec-4f82-8886-96c49d552e9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:11:53.623063 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d579f1f-34ec-4f82-8886-96c49d552e9c admin admin] PATH_INFO: `/v3/users` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:11:53.623729 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d579f1f-34ec-4f82-8886-96c49d552e9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:11:53.625681 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d579f1f-34ec-4f82-8886-96c49d552e9c admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:11:53.703978 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1021/2005] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:11:53 2022] POST /identity/v3/users => generated 287 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:11:53.708031 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce586c17-48a5-498e-8330-265fa9e8b48a admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:11:53.719288 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce586c17-48a5-498e-8330-265fa9e8b48a admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:11:53.724016 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b7df4c-12af-45c4-8d10-5d1036ddce15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:11:53.724620 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b7df4c-12af-45c4-8d10-5d1036ddce15 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:11:53.724905 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b7df4c-12af-45c4-8d10-5d1036ddce15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:11:53.725213 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b7df4c-12af-45c4-8d10-5d1036ddce15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:11:53.725894 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b7df4c-12af-45c4-8d10-5d1036ddce15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:11:53.727494 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b7df4c-12af-45c4-8d10-5d1036ddce15 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:11:53.733008 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:11:53.738500 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 985/2006] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:11:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:11:53.742500 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d579f1f-34ec-4f82-8886-96c49d552e9c admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:11:53.752467 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d579f1f-34ec-4f82-8886-96c49d552e9c admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:11:53.757910 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb7ba24-1c5d-4189-ac93-a30c7311de02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:11:53.758497 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb7ba24-1c5d-4189-ac93-a30c7311de02 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:11:53.758773 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb7ba24-1c5d-4189-ac93-a30c7311de02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:11:53.759045 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb7ba24-1c5d-4189-ac93-a30c7311de02 admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:11:53.759679 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb7ba24-1c5d-4189-ac93-a30c7311de02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:11:53.761311 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb7ba24-1c5d-4189-ac93-a30c7311de02 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:11:53.772566 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1022/2007] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:11:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:11:53.776627 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b7df4c-12af-45c4-8d10-5d1036ddce15 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:11:53.787283 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b7df4c-12af-45c4-8d10-5d1036ddce15 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:11:53.791855 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaebd290-2e45-4bbd-b04f-ba56d92a86ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:11:53.792498 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaebd290-2e45-4bbd-b04f-ba56d92a86ff admin admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:11:53.792781 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaebd290-2e45-4bbd-b04f-ba56d92a86ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:11:53.793061 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaebd290-2e45-4bbd-b04f-ba56d92a86ff admin admin] PATH_INFO: `/v3/projects/bb2534240ab74afbae7843231469486a/users/e7ee85ee112042f1abe5962d06d64ce0/roles/9bd0470b544240209a420f640c79c06c` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:11:53.807165 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaebd290-2e45-4bbd-b04f-ba56d92a86ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb2534240ab74afbae7843231469486a, user_id=e7ee85ee112042f1abe5962d06d64ce0, role_id=9bd0470b544240209a420f640c79c06c)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:11:53.809833 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaebd290-2e45-4bbd-b04f-ba56d92a86ff admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:11:53.903084 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 986/2008] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:11:53 2022] PUT /identity/v3/projects/bb2534240ab74afbae7843231469486a/users/e7ee85ee112042f1abe5962d06d64ce0/roles/9bd0470b544240209a420f640c79c06c => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:11:53.913998 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5af712f-77b9-41f1-921b-2c97343e2959 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:11:53.914287 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5af712f-77b9-41f1-921b-2c97343e2959 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:11:53.914565 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5af712f-77b9-41f1-921b-2c97343e2959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:11:53.941928 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-c5af712f-77b9-41f1-921b-2c97343e2959 None None] MFA Rules not processed for user `e7ee85ee112042f1abe5962d06d64ce0`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:11:53.943118 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-c5af712f-77b9-41f1-921b-2c97343e2959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:11:54.006009 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1023/2009] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:11:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 03:11:54.017331 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaebd290-2e45-4bbd-b04f-ba56d92a86ff admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:11:54.019175 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-eaebd290-2e45-4bbd-b04f-ba56d92a86ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:11:54.075504 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaebd290-2e45-4bbd-b04f-ba56d92a86ff admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:11:54.079580 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5f119d-c2f7-4734-8b89-44b029c7e6d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab599d98b0624a2e89090312bcab713f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:11:54.080175 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5f119d-c2f7-4734-8b89-44b029c7e6d4 service cinder] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:11:54.080454 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5f119d-c2f7-4734-8b89-44b029c7e6d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:11:54.080723 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5f119d-c2f7-4734-8b89-44b029c7e6d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:11:54.088827 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5f119d-c2f7-4734-8b89-44b029c7e6d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:11:54.090536 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5f119d-c2f7-4734-8b89-44b029c7e6d4 service cinder] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:11:54.099719 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 987/2010] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:11:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:11:55.821977 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005b282c-8f04-43d3-b62d-06d69a2e39e6 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:11:55.821977 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005b282c-8f04-43d3-b62d-06d69a2e39e6 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:11:55.821977 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005b282c-8f04-43d3-b62d-06d69a2e39e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:11:55.843941 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-005b282c-8f04-43d3-b62d-06d69a2e39e6 None None] MFA Rules not processed for user `e7ee85ee112042f1abe5962d06d64ce0`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:11:55.845545 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-005b282c-8f04-43d3-b62d-06d69a2e39e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:11:55.856919 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1024/2011] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:11:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:12:04.095788 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=36,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:12:04.364633 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c46d51-0e88-492e-8b5c-ab2efe69977e None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:12:04.364948 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c46d51-0e88-492e-8b5c-ab2efe69977e None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:12:04.365224 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c46d51-0e88-492e-8b5c-ab2efe69977e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:12:04.385034 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-09c46d51-0e88-492e-8b5c-ab2efe69977e None None] MFA Rules not processed for user `7a4ea9e3987e4f0b9ad9ebe6d3c69b7b`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:12:04.386275 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-09c46d51-0e88-492e-8b5c-ab2efe69977e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:12:04.443504 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 988/2012] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:12:04 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:12:05.837816 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=61,keystone:INSERT=3 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:12:07.227774 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005b282c-8f04-43d3-b62d-06d69a2e39e6 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:12:07.229225 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-005b282c-8f04-43d3-b62d-06d69a2e39e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:12:07.298760 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005b282c-8f04-43d3-b62d-06d69a2e39e6 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:12:07.303431 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f81a7a9-caab-4bcd-ba54-5e2244f76284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:12:07.304093 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f81a7a9-caab-4bcd-ba54-5e2244f76284 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:12:07.304374 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f81a7a9-caab-4bcd-ba54-5e2244f76284 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:12:07.304648 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f81a7a9-caab-4bcd-ba54-5e2244f76284 admin admin] PATH_INFO: `/v3/users/ced832cf86304814b020de4ed3e0955f` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:12:07.305990 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f81a7a9-caab-4bcd-ba54-5e2244f76284 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ced832cf86304814b020de4ed3e0955f)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:12:07.308157 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f81a7a9-caab-4bcd-ba54-5e2244f76284 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:12:07.538315 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-5f81a7a9-caab-4bcd-ba54-5e2244f76284 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ced832cf86304814b020de4ed3e0955f'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:12:07.538906 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-5f81a7a9-caab-4bcd-ba54-5e2244f76284 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ced832cf86304814b020de4ed3e0955f'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:12:07.621448 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-5f81a7a9-caab-4bcd-ba54-5e2244f76284 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ced832cf86304814b020de4ed3e0955f'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:12:07.622098 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-5f81a7a9-caab-4bcd-ba54-5e2244f76284 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ced832cf86304814b020de4ed3e0955f'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:12:07.630568 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-5f81a7a9-caab-4bcd-ba54-5e2244f76284 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ced832cf86304814b020de4ed3e0955f'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:12:07.648571 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1025/2013] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:12:07 2022] DELETE /identity/v3/users/ced832cf86304814b020de4ed3e0955f => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:12:07.887904 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c46d51-0e88-492e-8b5c-ab2efe69977e None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:12:07.902642 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c46d51-0e88-492e-8b5c-ab2efe69977e None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:12:07.959281 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28471c06-2207-4fa4-b167-11ac2aafcd18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:12:07.959925 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28471c06-2207-4fa4-b167-11ac2aafcd18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:12:07.960212 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28471c06-2207-4fa4-b167-11ac2aafcd18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:12:07.960487 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28471c06-2207-4fa4-b167-11ac2aafcd18 admin admin] PATH_INFO: `/v3/projects/9e3dfb4977504552a5480cd375ded7dd` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:12:07.961984 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28471c06-2207-4fa4-b167-11ac2aafcd18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e3dfb4977504552a5480cd375ded7dd)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:12:07.963597 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28471c06-2207-4fa4-b167-11ac2aafcd18 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:12:08.103668 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-28471c06-2207-4fa4-b167-11ac2aafcd18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e3dfb4977504552a5480cd375ded7dd', 'request_id': 'req-28471c06-2207-4fa4-b167-11ac2aafcd18'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:12:08.111553 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-28471c06-2207-4fa4-b167-11ac2aafcd18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e3dfb4977504552a5480cd375ded7dd', 'request_id': 'req-28471c06-2207-4fa4-b167-11ac2aafcd18'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:12:08.194078 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-28471c06-2207-4fa4-b167-11ac2aafcd18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e3dfb4977504552a5480cd375ded7dd', 'request_id': 'req-28471c06-2207-4fa4-b167-11ac2aafcd18'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:12:08.211482 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-28471c06-2207-4fa4-b167-11ac2aafcd18 admin admin] The token cache is being invalidate because project 9e3dfb4977504552a5480cd375ded7dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:12:08.211868 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-28471c06-2207-4fa4-b167-11ac2aafcd18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:12:08.212729 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-28471c06-2207-4fa4-b167-11ac2aafcd18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:12:08.217853 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 989/2014] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:12:07 2022] DELETE /identity/v3/projects/9e3dfb4977504552a5480cd375ded7dd => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:12:08.219859 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d1846b-0935-4085-a563-b907fbc359d5 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:12:08.220898 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d1846b-0935-4085-a563-b907fbc359d5 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:12:08.221208 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d1846b-0935-4085-a563-b907fbc359d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:12:08.246628 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-61d1846b-0935-4085-a563-b907fbc359d5 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:12:08.246628 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-61d1846b-0935-4085-a563-b907fbc359d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:12:08.307418 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1026/2015] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:12:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:12:08.373985 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640ea3bd-9da7-4456-93da-3a8582cbee93 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:12:08.374202 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640ea3bd-9da7-4456-93da-3a8582cbee93 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:12:08.374455 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640ea3bd-9da7-4456-93da-3a8582cbee93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:12:08.394742 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-640ea3bd-9da7-4456-93da-3a8582cbee93 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:12:08.395976 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-640ea3bd-9da7-4456-93da-3a8582cbee93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:12:08.404087 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 990/2016] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:12:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:12:08.408896 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d1846b-0935-4085-a563-b907fbc359d5 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:12:08.420205 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d1846b-0935-4085-a563-b907fbc359d5 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:12:08.425664 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b3c348-0f01-4031-9675-e0ae3e875e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:12:08.426306 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b3c348-0f01-4031-9675-e0ae3e875e33 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:12:08.426594 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b3c348-0f01-4031-9675-e0ae3e875e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:12:08.426904 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b3c348-0f01-4031-9675-e0ae3e875e33 admin admin] PATH_INFO: `/v3/domains` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:12:08.427610 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b3c348-0f01-4031-9675-e0ae3e875e33 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 03:12:08.427922 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b3c348-0f01-4031-9675-e0ae3e875e33 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:12:08.429618 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b3c348-0f01-4031-9675-e0ae3e875e33 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:12:08.444904 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1027/2017] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Oct 14 03:12:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 03:12:08.446497 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640ea3bd-9da7-4456-93da-3a8582cbee93 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:12:08.456461 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640ea3bd-9da7-4456-93da-3a8582cbee93 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:12:08.461197 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae14025-818e-4bfd-a11a-ebda334a1025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:12:08.461828 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae14025-818e-4bfd-a11a-ebda334a1025 admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:12:08.462110 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae14025-818e-4bfd-a11a-ebda334a1025 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:12:08.462380 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae14025-818e-4bfd-a11a-ebda334a1025 admin admin] PATH_INFO: `/v3/projects` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:12:08.463015 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae14025-818e-4bfd-a11a-ebda334a1025 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:12:08.464912 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae14025-818e-4bfd-a11a-ebda334a1025 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:12:08.568584 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 991/2018] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:12:08 2022] POST /identity/v3/projects => generated 399 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:12:08.572230 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b3c348-0f01-4031-9675-e0ae3e875e33 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:12:08.582451 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b3c348-0f01-4031-9675-e0ae3e875e33 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:12:08.586046 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:12:08.641879 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fa4c59-ad6a-44fd-ad46-ad3f3be1233e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:12:08.642423 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fa4c59-ad6a-44fd-ad46-ad3f3be1233e admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:12:08.642706 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fa4c59-ad6a-44fd-ad46-ad3f3be1233e admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:12:08.642982 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fa4c59-ad6a-44fd-ad46-ad3f3be1233e admin admin] PATH_INFO: `/v3/users` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:12:08.643613 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fa4c59-ad6a-44fd-ad46-ad3f3be1233e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:12:08.645530 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fa4c59-ad6a-44fd-ad46-ad3f3be1233e admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:12:08.712480 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1028/2019] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:12:08 2022] POST /identity/v3/users => generated 300 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:12:08.713749 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae14025-818e-4bfd-a11a-ebda334a1025 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:12:08.725570 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae14025-818e-4bfd-a11a-ebda334a1025 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:12:08.730682 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8217b4-fae1-4b3b-9672-2ae2aa385c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:12:08.731392 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8217b4-fae1-4b3b-9672-2ae2aa385c19 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:12:08.731747 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8217b4-fae1-4b3b-9672-2ae2aa385c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:12:08.732097 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8217b4-fae1-4b3b-9672-2ae2aa385c19 admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:12:08.732750 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8217b4-fae1-4b3b-9672-2ae2aa385c19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:12:08.734659 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8217b4-fae1-4b3b-9672-2ae2aa385c19 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:12:08.736059 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:12:08.745815 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 992/2020] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:12:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:12:08.756732 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fa4c59-ad6a-44fd-ad46-ad3f3be1233e admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:12:08.767447 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fa4c59-ad6a-44fd-ad46-ad3f3be1233e admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:12:08.772123 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fef0df-3418-474c-80e1-c4ec32a03dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:12:08.772716 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fef0df-3418-474c-80e1-c4ec32a03dfc admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:12:08.772996 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fef0df-3418-474c-80e1-c4ec32a03dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:12:08.773271 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fef0df-3418-474c-80e1-c4ec32a03dfc admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:12:08.773947 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fef0df-3418-474c-80e1-c4ec32a03dfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:12:08.775580 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fef0df-3418-474c-80e1-c4ec32a03dfc admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:12:08.786281 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1029/2021] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:12:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:12:08.790251 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8217b4-fae1-4b3b-9672-2ae2aa385c19 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:12:08.799989 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8217b4-fae1-4b3b-9672-2ae2aa385c19 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:12:08.804753 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092dc575-619a-43dc-b813-37a33a9fd289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:12:08.805406 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092dc575-619a-43dc-b813-37a33a9fd289 admin admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:12:08.805687 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092dc575-619a-43dc-b813-37a33a9fd289 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:12:08.806089 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092dc575-619a-43dc-b813-37a33a9fd289 admin admin] PATH_INFO: `/v3/projects/f54275a1e9c74a44905be3fac8c70dfd/users/a2fb5a1d050249f9bb7ed561505f6bf5/roles/9bd0470b544240209a420f640c79c06c` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:12:08.820740 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092dc575-619a-43dc-b813-37a33a9fd289 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f54275a1e9c74a44905be3fac8c70dfd, user_id=a2fb5a1d050249f9bb7ed561505f6bf5, role_id=9bd0470b544240209a420f640c79c06c)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:12:08.823438 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092dc575-619a-43dc-b813-37a33a9fd289 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:12:08.938056 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 993/2022] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:12:08 2022] PUT /identity/v3/projects/f54275a1e9c74a44905be3fac8c70dfd/users/a2fb5a1d050249f9bb7ed561505f6bf5/roles/9bd0470b544240209a420f640c79c06c => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:12:08.948066 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e5be23-ec2e-410a-b5ca-ac97218777ce None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:12:08.948350 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e5be23-ec2e-410a-b5ca-ac97218777ce None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:12:08.948622 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e5be23-ec2e-410a-b5ca-ac97218777ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:12:08.978409 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-33e5be23-ec2e-410a-b5ca-ac97218777ce None None] MFA Rules not processed for user `a2fb5a1d050249f9bb7ed561505f6bf5`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:12:08.979601 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-33e5be23-ec2e-410a-b5ca-ac97218777ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:12:09.046894 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1030/2023] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:12:08 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:12:09.058377 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092dc575-619a-43dc-b813-37a33a9fd289 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:12:09.060115 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-092dc575-619a-43dc-b813-37a33a9fd289 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:12:09.116594 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092dc575-619a-43dc-b813-37a33a9fd289 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:12:09.120373 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed58e0ec-a00d-42b1-ac34-e7fe8f030368 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab599d98b0624a2e89090312bcab713f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:12:09.120978 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed58e0ec-a00d-42b1-ac34-e7fe8f030368 service cinder] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:12:09.121262 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed58e0ec-a00d-42b1-ac34-e7fe8f030368 service cinder] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:12:09.121537 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed58e0ec-a00d-42b1-ac34-e7fe8f030368 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:12:09.130402 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed58e0ec-a00d-42b1-ac34-e7fe8f030368 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:12:09.131824 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed58e0ec-a00d-42b1-ac34-e7fe8f030368 service cinder] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:12:09.140725 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 994/2024] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:12:09 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:12:18.052471 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e5be23-ec2e-410a-b5ca-ac97218777ce None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:12:18.115377 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e5be23-ec2e-410a-b5ca-ac97218777ce None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:12:18.119901 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aafbfc3-7c98-4a72-aeb4-9b4119417f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:12:18.120532 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aafbfc3-7c98-4a72-aeb4-9b4119417f00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:12:18.120815 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aafbfc3-7c98-4a72-aeb4-9b4119417f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:12:18.121084 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aafbfc3-7c98-4a72-aeb4-9b4119417f00 admin admin] PATH_INFO: `/v3/users/a2fb5a1d050249f9bb7ed561505f6bf5` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:12:18.122268 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aafbfc3-7c98-4a72-aeb4-9b4119417f00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2fb5a1d050249f9bb7ed561505f6bf5)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:12:18.124195 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aafbfc3-7c98-4a72-aeb4-9b4119417f00 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:12:18.140738 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:12:18.292323 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-8aafbfc3-7c98-4a72-aeb4-9b4119417f00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2fb5a1d050249f9bb7ed561505f6bf5'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:12:18.293010 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-8aafbfc3-7c98-4a72-aeb4-9b4119417f00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2fb5a1d050249f9bb7ed561505f6bf5'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:12:18.352718 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-8aafbfc3-7c98-4a72-aeb4-9b4119417f00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2fb5a1d050249f9bb7ed561505f6bf5'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:12:18.353458 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-8aafbfc3-7c98-4a72-aeb4-9b4119417f00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2fb5a1d050249f9bb7ed561505f6bf5'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:12:18.362143 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-8aafbfc3-7c98-4a72-aeb4-9b4119417f00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2fb5a1d050249f9bb7ed561505f6bf5'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:12:18.381284 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1031/2025] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:12:18 2022] DELETE /identity/v3/users/a2fb5a1d050249f9bb7ed561505f6bf5 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:12:18.385317 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab07060-f832-4c59-b75a-e2130af585fa None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:12:18.385838 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab07060-f832-4c59-b75a-e2130af585fa None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:12:18.385935 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab07060-f832-4c59-b75a-e2130af585fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:12:18.404946 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-6ab07060-f832-4c59-b75a-e2130af585fa None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:12:18.406219 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-6ab07060-f832-4c59-b75a-e2130af585fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:12:18.472796 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 995/2026] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:12:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:12:18.479958 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aafbfc3-7c98-4a72-aeb4-9b4119417f00 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:12:18.481727 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-8aafbfc3-7c98-4a72-aeb4-9b4119417f00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:12:18.551419 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aafbfc3-7c98-4a72-aeb4-9b4119417f00 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:12:18.555465 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c9f452-3518-49de-a951-c5373da3e842 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:12:18.556150 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c9f452-3518-49de-a951-c5373da3e842 service neutron] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:12:18.556501 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c9f452-3518-49de-a951-c5373da3e842 service neutron] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:12:18.556885 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c9f452-3518-49de-a951-c5373da3e842 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:12:18.569625 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c9f452-3518-49de-a951-c5373da3e842 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:12:18.571109 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c9f452-3518-49de-a951-c5373da3e842 service neutron] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:12:18.583954 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1032/2027] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:12:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:12:19.145872 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab07060-f832-4c59-b75a-e2130af585fa None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:12:19.147341 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-6ab07060-f832-4c59-b75a-e2130af585fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:12:19.160284 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab07060-f832-4c59-b75a-e2130af585fa None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:12:19.166364 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68bc1d8-ff83-4ea2-849b-d0bbd15b6a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:12:19.167010 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68bc1d8-ff83-4ea2-849b-d0bbd15b6a3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:12:19.169170 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68bc1d8-ff83-4ea2-849b-d0bbd15b6a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:12:19.169170 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68bc1d8-ff83-4ea2-849b-d0bbd15b6a3a admin admin] PATH_INFO: `/v3/projects/f54275a1e9c74a44905be3fac8c70dfd` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:12:19.169170 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68bc1d8-ff83-4ea2-849b-d0bbd15b6a3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f54275a1e9c74a44905be3fac8c70dfd)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:12:19.171000 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68bc1d8-ff83-4ea2-849b-d0bbd15b6a3a admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:12:19.327439 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-c68bc1d8-ff83-4ea2-849b-d0bbd15b6a3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f54275a1e9c74a44905be3fac8c70dfd', 'request_id': 'req-c68bc1d8-ff83-4ea2-849b-d0bbd15b6a3a'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:12:19.334512 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-c68bc1d8-ff83-4ea2-849b-d0bbd15b6a3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f54275a1e9c74a44905be3fac8c70dfd', 'request_id': 'req-c68bc1d8-ff83-4ea2-849b-d0bbd15b6a3a'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:12:19.432935 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-c68bc1d8-ff83-4ea2-849b-d0bbd15b6a3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f54275a1e9c74a44905be3fac8c70dfd', 'request_id': 'req-c68bc1d8-ff83-4ea2-849b-d0bbd15b6a3a'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:12:19.440219 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:12:19.445216 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-c68bc1d8-ff83-4ea2-849b-d0bbd15b6a3a admin admin] The token cache is being invalidate because project f54275a1e9c74a44905be3fac8c70dfd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:12:19.445512 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-c68bc1d8-ff83-4ea2-849b-d0bbd15b6a3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:12:19.446185 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-c68bc1d8-ff83-4ea2-849b-d0bbd15b6a3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:12:19.458232 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 996/2028] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:12:19 2022] DELETE /identity/v3/projects/f54275a1e9c74a44905be3fac8c70dfd => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:12:21.066929 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb5de5-0959-44e4-bae8-544bef1ed8bf None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:12:21.067452 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb5de5-0959-44e4-bae8-544bef1ed8bf None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:12:21.067452 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb5de5-0959-44e4-bae8-544bef1ed8bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:12:21.087019 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-b1eb5de5-0959-44e4-bae8-544bef1ed8bf None None] MFA Rules not processed for user `44b37050076b42e6af1a67bd7d6d06e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:12:21.088208 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-b1eb5de5-0959-44e4-bae8-544bef1ed8bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:12:21.144564 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1033/2029] 10.222.0.39 () {58 vars in 1009 bytes} [Fri Oct 14 03:12:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:12:21.156756 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68bc1d8-ff83-4ea2-849b-d0bbd15b6a3a admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:12:21.158233 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-c68bc1d8-ff83-4ea2-849b-d0bbd15b6a3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:12:21.229724 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68bc1d8-ff83-4ea2-849b-d0bbd15b6a3a admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:12:21.235375 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e049b992-a3f5-4256-83f5-ac8a7d8ce915 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5e07da64804243bc18862072b027a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:12:21.236019 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e049b992-a3f5-4256-83f5-ac8a7d8ce915 service swift] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:12:21.236302 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e049b992-a3f5-4256-83f5-ac8a7d8ce915 service swift] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:12:21.236581 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e049b992-a3f5-4256-83f5-ac8a7d8ce915 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:12:21.245585 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e049b992-a3f5-4256-83f5-ac8a7d8ce915 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:12:21.247240 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e049b992-a3f5-4256-83f5-ac8a7d8ce915 service swift] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:12:21.256418 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 997/2030] 10.222.0.39 () {60 vars in 1384 bytes} [Fri Oct 14 03:12:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 14 03:12:22.124285 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fa5a26-4f25-497d-9737-edf3e84a9ea9 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:12:22.124768 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fa5a26-4f25-497d-9737-edf3e84a9ea9 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:12:22.124861 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fa5a26-4f25-497d-9737-edf3e84a9ea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:12:22.128161 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053c632d-8e29-4c04-87cc-f91ae03ce397 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:12:22.128447 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053c632d-8e29-4c04-87cc-f91ae03ce397 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:12:22.130608 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053c632d-8e29-4c04-87cc-f91ae03ce397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:12:22.145066 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-42fa5a26-4f25-497d-9737-edf3e84a9ea9 None None] MFA Rules not processed for user `44b37050076b42e6af1a67bd7d6d06e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:12:22.146317 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-42fa5a26-4f25-497d-9737-edf3e84a9ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:12:22.148806 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-053c632d-8e29-4c04-87cc-f91ae03ce397 None None] MFA Rules not processed for user `ab344e29f747423fb8766f2d8fbfef0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:12:22.150000 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-053c632d-8e29-4c04-87cc-f91ae03ce397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:12:22.153503 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1034/2031] 10.222.0.39 () {58 vars in 967 bytes} [Fri Oct 14 03:12:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:12:22.167369 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fa5a26-4f25-497d-9737-edf3e84a9ea9 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:12:22.179411 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fa5a26-4f25-497d-9737-edf3e84a9ea9 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:12:22.184377 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b86698e-dbdb-48ef-98fe-66ead674d3b1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5e07da64804243bc18862072b027a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:12:22.184970 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b86698e-dbdb-48ef-98fe-66ead674d3b1 service swift] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:12:22.185247 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b86698e-dbdb-48ef-98fe-66ead674d3b1 service swift] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:12:22.185519 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b86698e-dbdb-48ef-98fe-66ead674d3b1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:12:22.193580 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b86698e-dbdb-48ef-98fe-66ead674d3b1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:12:22.195198 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b86698e-dbdb-48ef-98fe-66ead674d3b1 service swift] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:12:22.203340 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1035/2032] 10.222.0.39 () {60 vars in 1384 bytes} [Fri Oct 14 03:12:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 14 03:12:22.208173 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 998/2033] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:12:22 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 03:12:22.834491 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17a908f-cc40-4d9f-af30-343dba70b88c None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:12:22.834778 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17a908f-cc40-4d9f-af30-343dba70b88c None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:12:22.835058 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17a908f-cc40-4d9f-af30-343dba70b88c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:12:22.854340 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-d17a908f-cc40-4d9f-af30-343dba70b88c None None] MFA Rules not processed for user `44b37050076b42e6af1a67bd7d6d06e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:12:22.855557 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-d17a908f-cc40-4d9f-af30-343dba70b88c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:12:22.863435 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1036/2034] 10.222.0.39 () {58 vars in 1009 bytes} [Fri Oct 14 03:12:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:12:22.875272 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053c632d-8e29-4c04-87cc-f91ae03ce397 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:12:22.887918 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053c632d-8e29-4c04-87cc-f91ae03ce397 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:12:22.893101 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffa8972-a9a9-415c-bcbb-58c839b9c349 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5e07da64804243bc18862072b027a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:12:22.893719 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffa8972-a9a9-415c-bcbb-58c839b9c349 service swift] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:12:22.894011 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffa8972-a9a9-415c-bcbb-58c839b9c349 service swift] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:12:22.894285 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffa8972-a9a9-415c-bcbb-58c839b9c349 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:12:22.903590 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffa8972-a9a9-415c-bcbb-58c839b9c349 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:12:22.905496 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffa8972-a9a9-415c-bcbb-58c839b9c349 service swift] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:12:22.914891 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 999/2035] 10.222.0.39 () {60 vars in 1384 bytes} [Fri Oct 14 03:12:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 14 03:12:23.558796 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7735eae3-53eb-4d2d-b33c-1d608fbe0b63 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:12:23.559360 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7735eae3-53eb-4d2d-b33c-1d608fbe0b63 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:12:23.559465 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7735eae3-53eb-4d2d-b33c-1d608fbe0b63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:12:23.578815 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-7735eae3-53eb-4d2d-b33c-1d608fbe0b63 None None] MFA Rules not processed for user `44b37050076b42e6af1a67bd7d6d06e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:12:23.580023 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-7735eae3-53eb-4d2d-b33c-1d608fbe0b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:12:23.587140 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1037/2036] 10.222.0.39 () {58 vars in 967 bytes} [Fri Oct 14 03:12:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:12:23.599586 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffa8972-a9a9-415c-bcbb-58c839b9c349 service swift] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:12:23.611395 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffa8972-a9a9-415c-bcbb-58c839b9c349 service swift] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:12:23.616480 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba894e5-573d-4fe4-a57e-d502de8849ce service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5e07da64804243bc18862072b027a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:12:23.617067 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba894e5-573d-4fe4-a57e-d502de8849ce service swift] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:12:23.617350 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba894e5-573d-4fe4-a57e-d502de8849ce service swift] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:12:23.617627 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba894e5-573d-4fe4-a57e-d502de8849ce service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:12:23.626010 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba894e5-573d-4fe4-a57e-d502de8849ce service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:12:23.627582 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba894e5-573d-4fe4-a57e-d502de8849ce service swift] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:12:23.636181 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1000/2037] 10.222.0.39 () {60 vars in 1384 bytes} [Fri Oct 14 03:12:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 14 03:12:25.226250 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7735eae3-53eb-4d2d-b33c-1d608fbe0b63 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:12:25.227862 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-7735eae3-53eb-4d2d-b33c-1d608fbe0b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:12:25.240313 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:12:25.293601 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7735eae3-53eb-4d2d-b33c-1d608fbe0b63 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:12:25.297256 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8b4d3f-2df3-4d4b-b4ab-ece25af90453 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:12:25.297888 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8b4d3f-2df3-4d4b-b4ab-ece25af90453 service neutron] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:12:25.298175 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8b4d3f-2df3-4d4b-b4ab-ece25af90453 service neutron] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:12:25.298455 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8b4d3f-2df3-4d4b-b4ab-ece25af90453 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:12:25.302765 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-ef8b4d3f-2df3-4d4b-b4ab-ece25af90453 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:12:25.377367 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8b4d3f-2df3-4d4b-b4ab-ece25af90453 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:12:25.379615 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8b4d3f-2df3-4d4b-b4ab-ece25af90453 service neutron] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:12:25.392441 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1038/2038] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:12:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:12:26.237873 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba894e5-573d-4fe4-a57e-d502de8849ce service swift] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:12:26.239436 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-0ba894e5-573d-4fe4-a57e-d502de8849ce service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:12:26.304656 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba894e5-573d-4fe4-a57e-d502de8849ce service swift] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:12:26.310341 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5ff2e9-8e06-4c6d-8333-a7ab21c4bbd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:12:26.311511 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5ff2e9-8e06-4c6d-8333-a7ab21c4bbd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:12:26.311511 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5ff2e9-8e06-4c6d-8333-a7ab21c4bbd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:12:26.311727 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5ff2e9-8e06-4c6d-8333-a7ab21c4bbd3 admin admin] PATH_INFO: `/v3/users/e7ee85ee112042f1abe5962d06d64ce0` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:12:26.312676 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5ff2e9-8e06-4c6d-8333-a7ab21c4bbd3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7ee85ee112042f1abe5962d06d64ce0)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:12:26.314701 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5ff2e9-8e06-4c6d-8333-a7ab21c4bbd3 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:12:26.320831 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:12:26.483191 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-3b5ff2e9-8e06-4c6d-8333-a7ab21c4bbd3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7ee85ee112042f1abe5962d06d64ce0'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:12:26.491983 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-3b5ff2e9-8e06-4c6d-8333-a7ab21c4bbd3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7ee85ee112042f1abe5962d06d64ce0'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:12:26.504345 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-3b5ff2e9-8e06-4c6d-8333-a7ab21c4bbd3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7ee85ee112042f1abe5962d06d64ce0'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:12:26.505200 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-3b5ff2e9-8e06-4c6d-8333-a7ab21c4bbd3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7ee85ee112042f1abe5962d06d64ce0'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:12:26.518658 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8b4d3f-2df3-4d4b-b4ab-ece25af90453 service neutron] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:12:26.527242 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8b4d3f-2df3-4d4b-b4ab-ece25af90453 service neutron] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:12:26.530891 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010e6c22-89a3-48b0-a531-1192cbb54197 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:12:26.531487 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010e6c22-89a3-48b0-a531-1192cbb54197 service neutron] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:12:26.531764 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010e6c22-89a3-48b0-a531-1192cbb54197 service neutron] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:12:26.532036 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010e6c22-89a3-48b0-a531-1192cbb54197 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:12:26.534285 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-010e6c22-89a3-48b0-a531-1192cbb54197 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:12:26.548033 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010e6c22-89a3-48b0-a531-1192cbb54197 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:12:26.550120 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010e6c22-89a3-48b0-a531-1192cbb54197 service neutron] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:12:26.560970 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1039/2039] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:12:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 03:12:26.587311 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-3b5ff2e9-8e06-4c6d-8333-a7ab21c4bbd3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7ee85ee112042f1abe5962d06d64ce0'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:12:26.594673 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1001/2040] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:12:26 2022] DELETE /identity/v3/users/e7ee85ee112042f1abe5962d06d64ce0 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:12:26.599933 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cfc143-bf62-4ffe-a759-defd4860c8ae None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:12:26.600226 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cfc143-bf62-4ffe-a759-defd4860c8ae None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:12:26.600507 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cfc143-bf62-4ffe-a759-defd4860c8ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:12:26.621089 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-b2cfc143-bf62-4ffe-a759-defd4860c8ae None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:12:26.622584 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-b2cfc143-bf62-4ffe-a759-defd4860c8ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:12:26.697408 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1040/2041] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:12:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:12:26.704747 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5ff2e9-8e06-4c6d-8333-a7ab21c4bbd3 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:12:26.706895 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-3b5ff2e9-8e06-4c6d-8333-a7ab21c4bbd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:12:26.765175 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5ff2e9-8e06-4c6d-8333-a7ab21c4bbd3 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:12:26.769329 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84dd006-e612-43a8-a12f-9502cb0e9493 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:12:26.769966 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84dd006-e612-43a8-a12f-9502cb0e9493 service neutron] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:12:26.770251 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84dd006-e612-43a8-a12f-9502cb0e9493 service neutron] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:12:26.770528 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84dd006-e612-43a8-a12f-9502cb0e9493 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:12:26.779447 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84dd006-e612-43a8-a12f-9502cb0e9493 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:12:26.780911 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84dd006-e612-43a8-a12f-9502cb0e9493 service neutron] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:12:26.791330 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1002/2042] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:12:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:12:27.030449 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cfc143-bf62-4ffe-a759-defd4860c8ae None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:12:27.030449 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-b2cfc143-bf62-4ffe-a759-defd4860c8ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:12:27.043150 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cfc143-bf62-4ffe-a759-defd4860c8ae None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:12:27.056181 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3f5b44-e817-4273-a660-8b16693c8814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:12:27.059875 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3f5b44-e817-4273-a660-8b16693c8814 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:12:27.059875 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3f5b44-e817-4273-a660-8b16693c8814 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:12:27.059875 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3f5b44-e817-4273-a660-8b16693c8814 admin admin] PATH_INFO: `/v3/projects/bb2534240ab74afbae7843231469486a` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:12:27.059875 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3f5b44-e817-4273-a660-8b16693c8814 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb2534240ab74afbae7843231469486a)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:12:27.062292 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3f5b44-e817-4273-a660-8b16693c8814 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:12:27.164763 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:12:27.176587 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-ea3f5b44-e817-4273-a660-8b16693c8814 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb2534240ab74afbae7843231469486a', 'request_id': 'req-ea3f5b44-e817-4273-a660-8b16693c8814'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:12:27.182765 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-ea3f5b44-e817-4273-a660-8b16693c8814 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb2534240ab74afbae7843231469486a', 'request_id': 'req-ea3f5b44-e817-4273-a660-8b16693c8814'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:12:27.261432 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-ea3f5b44-e817-4273-a660-8b16693c8814 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb2534240ab74afbae7843231469486a', 'request_id': 'req-ea3f5b44-e817-4273-a660-8b16693c8814'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:12:27.273344 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-ea3f5b44-e817-4273-a660-8b16693c8814 admin admin] The token cache is being invalidate because project bb2534240ab74afbae7843231469486a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81517) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:12:27.273563 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-ea3f5b44-e817-4273-a660-8b16693c8814 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:12:27.274232 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-ea3f5b44-e817-4273-a660-8b16693c8814 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:12:27.275739 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1041/2043] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:12:27 2022] DELETE /identity/v3/projects/bb2534240ab74afbae7843231469486a => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:12:36.786659 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=49,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:12:37.284280 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=12,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:12:42.464184 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84dd006-e612-43a8-a12f-9502cb0e9493 service neutron] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:12:42.465980 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-f84dd006-e612-43a8-a12f-9502cb0e9493 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:12:42.525402 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84dd006-e612-43a8-a12f-9502cb0e9493 service neutron] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:12:42.581424 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cefd43-afea-4c6c-b0ff-350afe9e6171 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab599d98b0624a2e89090312bcab713f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:12:42.582174 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cefd43-afea-4c6c-b0ff-350afe9e6171 service cinder] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:12:42.582460 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cefd43-afea-4c6c-b0ff-350afe9e6171 service cinder] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:12:42.582734 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cefd43-afea-4c6c-b0ff-350afe9e6171 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:12:42.584532 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-55cefd43-afea-4c6c-b0ff-350afe9e6171 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:12:42.646587 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cefd43-afea-4c6c-b0ff-350afe9e6171 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:12:42.648113 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cefd43-afea-4c6c-b0ff-350afe9e6171 service cinder] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:12:42.657518 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1003/2044] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:12:42 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:12:43.123076 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d066d47e-62d5-43ec-8b9f-8b9196c54974 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:12:43.127526 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d066d47e-62d5-43ec-8b9f-8b9196c54974 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:12:43.127526 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d066d47e-62d5-43ec-8b9f-8b9196c54974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:12:43.143452 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-d066d47e-62d5-43ec-8b9f-8b9196c54974 None None] MFA Rules not processed for user `fb1c279c2f7f404ab9c846a3ba594886`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:12:43.144647 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-d066d47e-62d5-43ec-8b9f-8b9196c54974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:12:43.199310 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1042/2045] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:12:43 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:12:46.030616 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cefd43-afea-4c6c-b0ff-350afe9e6171 service cinder] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:12:46.032072 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-55cefd43-afea-4c6c-b0ff-350afe9e6171 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:12:46.080876 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d066d47e-62d5-43ec-8b9f-8b9196c54974 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:12:46.082506 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-d066d47e-62d5-43ec-8b9f-8b9196c54974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:12:46.089393 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cefd43-afea-4c6c-b0ff-350afe9e6171 service cinder] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:12:46.093161 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ec8e72-587e-4602-9441-d9d02bd6422d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:12:46.093786 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ec8e72-587e-4602-9441-d9d02bd6422d service neutron] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:12:46.094078 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ec8e72-587e-4602-9441-d9d02bd6422d service neutron] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:12:46.094398 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ec8e72-587e-4602-9441-d9d02bd6422d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:12:46.096251 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-f3ec8e72-587e-4602-9441-d9d02bd6422d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:12:46.107831 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ec8e72-587e-4602-9441-d9d02bd6422d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:12:46.109257 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ec8e72-587e-4602-9441-d9d02bd6422d service neutron] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:12:46.120257 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1004/2046] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:12:46 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:12:46.163261 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d066d47e-62d5-43ec-8b9f-8b9196c54974 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:12:46.168791 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3d93f1-c42b-4e87-ba73-0f4562330ece service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b07bf0faca47b28b50990d1720853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:12:46.169395 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3d93f1-c42b-4e87-ba73-0f4562330ece service nova] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:12:46.169676 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3d93f1-c42b-4e87-ba73-0f4562330ece service nova] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:12:46.169977 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3d93f1-c42b-4e87-ba73-0f4562330ece service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:12:46.178553 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3d93f1-c42b-4e87-ba73-0f4562330ece service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:12:46.180280 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3d93f1-c42b-4e87-ba73-0f4562330ece service nova] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:12:46.190532 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1043/2047] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:12:46 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:12:51.171178 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ec8e72-587e-4602-9441-d9d02bd6422d service neutron] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:12:51.172665 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-f3ec8e72-587e-4602-9441-d9d02bd6422d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:12:51.184574 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ec8e72-587e-4602-9441-d9d02bd6422d service neutron] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:12:51.188202 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6446099-c096-4e8b-a7bb-e54ff92b15df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab599d98b0624a2e89090312bcab713f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:12:51.188803 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6446099-c096-4e8b-a7bb-e54ff92b15df service cinder] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:12:51.189087 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6446099-c096-4e8b-a7bb-e54ff92b15df service cinder] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:12:51.189392 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6446099-c096-4e8b-a7bb-e54ff92b15df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:12:51.191159 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-d6446099-c096-4e8b-a7bb-e54ff92b15df service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:12:51.202446 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:12:51.263508 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6446099-c096-4e8b-a7bb-e54ff92b15df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:12:51.265988 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6446099-c096-4e8b-a7bb-e54ff92b15df service cinder] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:12:51.278547 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1005/2048] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:12:51 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:12:56.185188 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=59 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:12:58.866856 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3d93f1-c42b-4e87-ba73-0f4562330ece service nova] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:12:58.883395 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3d93f1-c42b-4e87-ba73-0f4562330ece service nova] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:12:58.888544 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a876840d-9310-468f-ba94-d4a6a5b064fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:12:58.889159 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a876840d-9310-468f-ba94-d4a6a5b064fd service neutron] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:12:58.889435 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a876840d-9310-468f-ba94-d4a6a5b064fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:12:58.889703 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a876840d-9310-468f-ba94-d4a6a5b064fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:12:58.891500 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-a876840d-9310-468f-ba94-d4a6a5b064fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:12:58.902095 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a876840d-9310-468f-ba94-d4a6a5b064fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:12:58.903617 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a876840d-9310-468f-ba94-d4a6a5b064fd service neutron] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:12:58.912893 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1044/2049] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:12:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:13:01.002368 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6446099-c096-4e8b-a7bb-e54ff92b15df service cinder] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:13:01.004486 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-d6446099-c096-4e8b-a7bb-e54ff92b15df service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:13:01.022915 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6446099-c096-4e8b-a7bb-e54ff92b15df service cinder] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:13:01.028624 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df6f699-b80c-4054-aebe-b06172f06204 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b07bf0faca47b28b50990d1720853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:13:01.029061 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df6f699-b80c-4054-aebe-b06172f06204 service nova] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:01.029338 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df6f699-b80c-4054-aebe-b06172f06204 service nova] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:01.029613 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df6f699-b80c-4054-aebe-b06172f06204 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:01.031415 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-9df6f699-b80c-4054-aebe-b06172f06204 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:13:01.045592 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df6f699-b80c-4054-aebe-b06172f06204 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:13:01.047422 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df6f699-b80c-4054-aebe-b06172f06204 service nova] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:13:01.060064 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1006/2050] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:13:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:13:04.814699 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ae718e-6271-4fe6-9927-e353d31dafa9 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:04.815305 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ae718e-6271-4fe6-9927-e353d31dafa9 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:04.815305 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ae718e-6271-4fe6-9927-e353d31dafa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:04.834955 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-f4ae718e-6271-4fe6-9927-e353d31dafa9 None None] MFA Rules not processed for user `410b7c9e93c94ce78d1b5c0e646307a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:13:04.836163 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-f4ae718e-6271-4fe6-9927-e353d31dafa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:13:04.843305 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1045/2051] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:13:04 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:13:05.146879 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df6f699-b80c-4054-aebe-b06172f06204 service nova] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:13:05.148412 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-9df6f699-b80c-4054-aebe-b06172f06204 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:13:05.159754 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df6f699-b80c-4054-aebe-b06172f06204 service nova] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:13:05.163798 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77df979a-3034-43b1-bda6-bec83583950a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:13:05.164418 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77df979a-3034-43b1-bda6-bec83583950a service neutron] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:05.164695 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77df979a-3034-43b1-bda6-bec83583950a service neutron] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:05.164965 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77df979a-3034-43b1-bda6-bec83583950a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:05.166654 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-77df979a-3034-43b1-bda6-bec83583950a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:13:05.175894 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77df979a-3034-43b1-bda6-bec83583950a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:13:05.177360 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77df979a-3034-43b1-bda6-bec83583950a service neutron] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:13:05.185740 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1007/2052] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:13:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:13:05.574164 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ae718e-6271-4fe6-9927-e353d31dafa9 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:13:05.579706 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-f4ae718e-6271-4fe6-9927-e353d31dafa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:13:05.646836 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ae718e-6271-4fe6-9927-e353d31dafa9 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:13:05.652687 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bae459-256e-4d26-be45-58fd518aab35 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '922ef7811aed450baff7f936011b0af1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:13:05.653295 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bae459-256e-4d26-be45-58fd518aab35 service placement] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:05.653580 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bae459-256e-4d26-be45-58fd518aab35 service placement] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:05.653890 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bae459-256e-4d26-be45-58fd518aab35 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:05.655631 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-03bae459-256e-4d26-be45-58fd518aab35 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:13:05.667656 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bae459-256e-4d26-be45-58fd518aab35 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:13:05.669245 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bae459-256e-4d26-be45-58fd518aab35 service placement] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:13:05.679037 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1046/2053] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:13:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:13:11.201262 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77df979a-3034-43b1-bda6-bec83583950a service neutron] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:13:11.222807 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77df979a-3034-43b1-bda6-bec83583950a service neutron] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:13:11.230461 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a244142-a9d7-432b-9f8f-017b0a72509c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b07bf0faca47b28b50990d1720853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:13:11.231472 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a244142-a9d7-432b-9f8f-017b0a72509c service nova] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:11.232006 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a244142-a9d7-432b-9f8f-017b0a72509c service nova] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:11.232498 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a244142-a9d7-432b-9f8f-017b0a72509c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:11.238678 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-9a244142-a9d7-432b-9f8f-017b0a72509c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:13:11.323268 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a244142-a9d7-432b-9f8f-017b0a72509c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:13:11.324920 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a244142-a9d7-432b-9f8f-017b0a72509c service nova] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:13:11.342010 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1008/2054] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:13:11 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:13:15.674876 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=41 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:13:18.947048 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bae459-256e-4d26-be45-58fd518aab35 service placement] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:13:18.958947 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bae459-256e-4d26-be45-58fd518aab35 service placement] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:13:18.964832 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe8548a-6995-490f-b9fb-f640595cf5c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b07bf0faca47b28b50990d1720853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:13:18.965419 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe8548a-6995-490f-b9fb-f640595cf5c3 service nova] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:18.965699 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe8548a-6995-490f-b9fb-f640595cf5c3 service nova] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:18.966009 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe8548a-6995-490f-b9fb-f640595cf5c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:18.967727 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-fbe8548a-6995-490f-b9fb-f640595cf5c3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:13:18.980256 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe8548a-6995-490f-b9fb-f640595cf5c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:13:18.981853 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe8548a-6995-490f-b9fb-f640595cf5c3 service nova] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:13:18.992514 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1047/2055] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:13:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:13:19.744566 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a244142-a9d7-432b-9f8f-017b0a72509c service nova] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:13:19.753596 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a244142-a9d7-432b-9f8f-017b0a72509c service nova] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:13:19.757492 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd1687f-df68-46f1-9d24-22890ad4a4de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:13:19.758798 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd1687f-df68-46f1-9d24-22890ad4a4de service neutron] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:19.759082 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd1687f-df68-46f1-9d24-22890ad4a4de service neutron] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:19.759357 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd1687f-df68-46f1-9d24-22890ad4a4de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:19.761147 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-2bd1687f-df68-46f1-9d24-22890ad4a4de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:13:19.823427 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd1687f-df68-46f1-9d24-22890ad4a4de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:13:19.824847 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd1687f-df68-46f1-9d24-22890ad4a4de service neutron] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:13:19.834269 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1009/2056] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:13:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:13:23.910542 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe8548a-6995-490f-b9fb-f640595cf5c3 service nova] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:13:23.910542 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-fbe8548a-6995-490f-b9fb-f640595cf5c3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:13:23.921828 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe8548a-6995-490f-b9fb-f640595cf5c3 service nova] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:13:23.929446 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0581de87-d534-4320-afb6-283aa2502e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:13:23.930498 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0581de87-d534-4320-afb6-283aa2502e57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:23.930972 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0581de87-d534-4320-afb6-283aa2502e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:23.931258 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0581de87-d534-4320-afb6-283aa2502e57 admin admin] PATH_INFO: `/v3/users/7a4ea9e3987e4f0b9ad9ebe6d3c69b7b` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:23.932521 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0581de87-d534-4320-afb6-283aa2502e57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a4ea9e3987e4f0b9ad9ebe6d3c69b7b)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:13:23.934520 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0581de87-d534-4320-afb6-283aa2502e57 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:13:24.113425 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-0581de87-d534-4320-afb6-283aa2502e57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a4ea9e3987e4f0b9ad9ebe6d3c69b7b'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:13:24.114271 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-0581de87-d534-4320-afb6-283aa2502e57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a4ea9e3987e4f0b9ad9ebe6d3c69b7b'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:13:24.171825 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-0581de87-d534-4320-afb6-283aa2502e57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a4ea9e3987e4f0b9ad9ebe6d3c69b7b'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:13:24.172599 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-0581de87-d534-4320-afb6-283aa2502e57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a4ea9e3987e4f0b9ad9ebe6d3c69b7b'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:13:24.184026 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-0581de87-d534-4320-afb6-283aa2502e57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a4ea9e3987e4f0b9ad9ebe6d3c69b7b'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:13:24.204072 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1048/2057] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:13:23 2022] DELETE /identity/v3/users/7a4ea9e3987e4f0b9ad9ebe6d3c69b7b => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:13:24.453931 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd1687f-df68-46f1-9d24-22890ad4a4de service neutron] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:13:24.459947 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=93 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:13:24.467443 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd1687f-df68-46f1-9d24-22890ad4a4de service neutron] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:13:24.535016 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1700e2e-6623-4371-9db3-921bcc9bf11e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:13:24.535974 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1700e2e-6623-4371-9db3-921bcc9bf11e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:24.536521 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1700e2e-6623-4371-9db3-921bcc9bf11e admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:24.536804 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1700e2e-6623-4371-9db3-921bcc9bf11e admin admin] PATH_INFO: `/v3/projects/44d8a8798bc64aefadd52c30a21002ef` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:24.538238 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1700e2e-6623-4371-9db3-921bcc9bf11e admin admin] RBAC: Authorizing `identity:delete_project(project_id=44d8a8798bc64aefadd52c30a21002ef)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:13:24.541860 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1700e2e-6623-4371-9db3-921bcc9bf11e admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:13:24.686079 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-e1700e2e-6623-4371-9db3-921bcc9bf11e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44d8a8798bc64aefadd52c30a21002ef', 'request_id': 'req-e1700e2e-6623-4371-9db3-921bcc9bf11e'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:13:24.692451 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-e1700e2e-6623-4371-9db3-921bcc9bf11e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44d8a8798bc64aefadd52c30a21002ef', 'request_id': 'req-e1700e2e-6623-4371-9db3-921bcc9bf11e'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:13:24.754587 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-e1700e2e-6623-4371-9db3-921bcc9bf11e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44d8a8798bc64aefadd52c30a21002ef', 'request_id': 'req-e1700e2e-6623-4371-9db3-921bcc9bf11e'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:13:24.767155 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-e1700e2e-6623-4371-9db3-921bcc9bf11e admin admin] The token cache is being invalidate because project 44d8a8798bc64aefadd52c30a21002ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:13:24.767444 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-e1700e2e-6623-4371-9db3-921bcc9bf11e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:13:24.768103 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-e1700e2e-6623-4371-9db3-921bcc9bf11e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:13:24.769616 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1010/2058] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:13:24 2022] DELETE /identity/v3/projects/44d8a8798bc64aefadd52c30a21002ef => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:13:24.774816 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea749c1-4553-4106-9b2c-6f3a8a174e15 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:24.775248 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea749c1-4553-4106-9b2c-6f3a8a174e15 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:24.775571 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea749c1-4553-4106-9b2c-6f3a8a174e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:24.798599 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-2ea749c1-4553-4106-9b2c-6f3a8a174e15 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:13:24.799944 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-2ea749c1-4553-4106-9b2c-6f3a8a174e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:13:24.869748 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1049/2059] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:13:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:13:24.877552 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baac284-fb11-4de6-a1f4-e0137e622027 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:24.877893 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baac284-fb11-4de6-a1f4-e0137e622027 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:24.878171 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baac284-fb11-4de6-a1f4-e0137e622027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:24.897627 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-3baac284-fb11-4de6-a1f4-e0137e622027 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:13:24.898888 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-3baac284-fb11-4de6-a1f4-e0137e622027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:13:24.907055 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1011/2060] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:13:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:13:24.911526 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea749c1-4553-4106-9b2c-6f3a8a174e15 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:13:24.922047 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea749c1-4553-4106-9b2c-6f3a8a174e15 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:13:24.926808 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eba4e3-9209-4d6b-b600-5477b74b60c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:13:24.927410 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eba4e3-9209-4d6b-b600-5477b74b60c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:24.927686 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eba4e3-9209-4d6b-b600-5477b74b60c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:24.927954 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eba4e3-9209-4d6b-b600-5477b74b60c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:24.928622 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51eba4e3-9209-4d6b-b600-5477b74b60c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 03:13:24.928926 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51eba4e3-9209-4d6b-b600-5477b74b60c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:13:24.931158 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51eba4e3-9209-4d6b-b600-5477b74b60c2 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:13:24.942912 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1050/2061] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Oct 14 03:13:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 03:13:24.947284 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baac284-fb11-4de6-a1f4-e0137e622027 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:13:24.956542 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baac284-fb11-4de6-a1f4-e0137e622027 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:13:24.962162 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce86e72-ba3f-4f14-8996-14cddfe1ddc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:13:24.962871 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce86e72-ba3f-4f14-8996-14cddfe1ddc6 admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:24.963146 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce86e72-ba3f-4f14-8996-14cddfe1ddc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:24.963412 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce86e72-ba3f-4f14-8996-14cddfe1ddc6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:24.964148 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce86e72-ba3f-4f14-8996-14cddfe1ddc6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:13:24.966047 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce86e72-ba3f-4f14-8996-14cddfe1ddc6 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:13:25.075604 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1012/2062] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:13:24 2022] POST /identity/v3/projects => generated 385 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:13:25.079922 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eba4e3-9209-4d6b-b600-5477b74b60c2 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:13:25.091297 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eba4e3-9209-4d6b-b600-5477b74b60c2 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:13:25.135149 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:13:25.147809 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b58c6a-154f-475d-98df-d7b0a6ee3b74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:13:25.148904 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b58c6a-154f-475d-98df-d7b0a6ee3b74 admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:25.149196 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b58c6a-154f-475d-98df-d7b0a6ee3b74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:25.149477 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b58c6a-154f-475d-98df-d7b0a6ee3b74 admin admin] PATH_INFO: `/v3/users` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:25.150137 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b58c6a-154f-475d-98df-d7b0a6ee3b74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:13:25.152058 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b58c6a-154f-475d-98df-d7b0a6ee3b74 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:13:25.240050 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1051/2063] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:13:25 2022] POST /identity/v3/users => generated 293 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:13:25.244575 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce86e72-ba3f-4f14-8996-14cddfe1ddc6 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:13:25.255282 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce86e72-ba3f-4f14-8996-14cddfe1ddc6 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:13:25.259868 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad8707e-094a-4132-a131-0ef3c295478a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:13:25.260453 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad8707e-094a-4132-a131-0ef3c295478a admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:25.260731 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad8707e-094a-4132-a131-0ef3c295478a admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:25.261004 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad8707e-094a-4132-a131-0ef3c295478a admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:25.261640 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad8707e-094a-4132-a131-0ef3c295478a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:13:25.263304 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad8707e-094a-4132-a131-0ef3c295478a admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:13:25.272882 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1013/2064] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:13:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:13:25.278641 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b58c6a-154f-475d-98df-d7b0a6ee3b74 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:13:25.288675 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b58c6a-154f-475d-98df-d7b0a6ee3b74 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:13:25.293865 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1421308f-0894-4d54-82b1-79d64b97731d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:13:25.294450 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1421308f-0894-4d54-82b1-79d64b97731d admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:25.294744 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1421308f-0894-4d54-82b1-79d64b97731d admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:25.294997 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1421308f-0894-4d54-82b1-79d64b97731d admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:25.295719 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1421308f-0894-4d54-82b1-79d64b97731d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:13:25.297313 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1421308f-0894-4d54-82b1-79d64b97731d admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:13:25.308632 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1052/2065] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:13:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:13:25.312682 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad8707e-094a-4132-a131-0ef3c295478a admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:13:25.323194 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad8707e-094a-4132-a131-0ef3c295478a admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:13:25.327731 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b7a5ae-460c-4cca-9139-d791b569d671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:13:25.328372 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b7a5ae-460c-4cca-9139-d791b569d671 admin admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:25.328649 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b7a5ae-460c-4cca-9139-d791b569d671 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:25.328919 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b7a5ae-460c-4cca-9139-d791b569d671 admin admin] PATH_INFO: `/v3/projects/9b612d10ff3b402ea7571f3134c88a11/users/30901704e6a7408cb7fdb8d5479dfd8a/roles/9bd0470b544240209a420f640c79c06c` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:25.343924 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b7a5ae-460c-4cca-9139-d791b569d671 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b612d10ff3b402ea7571f3134c88a11, user_id=30901704e6a7408cb7fdb8d5479dfd8a, role_id=9bd0470b544240209a420f640c79c06c)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:13:25.346549 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b7a5ae-460c-4cca-9139-d791b569d671 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:13:25.428329 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1014/2066] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:13:25 2022] PUT /identity/v3/projects/9b612d10ff3b402ea7571f3134c88a11/users/30901704e6a7408cb7fdb8d5479dfd8a/roles/9bd0470b544240209a420f640c79c06c => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:13:25.438455 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc9de55-cd56-4efa-a2a6-73dcd3d5ae4a None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:25.438749 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc9de55-cd56-4efa-a2a6-73dcd3d5ae4a None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:25.439160 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc9de55-cd56-4efa-a2a6-73dcd3d5ae4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:25.467476 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-8bc9de55-cd56-4efa-a2a6-73dcd3d5ae4a None None] MFA Rules not processed for user `30901704e6a7408cb7fdb8d5479dfd8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:13:25.468683 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-8bc9de55-cd56-4efa-a2a6-73dcd3d5ae4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:13:25.532656 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1053/2067] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:13:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:13:25.544064 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b7a5ae-460c-4cca-9139-d791b569d671 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:13:25.545788 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-e9b7a5ae-460c-4cca-9139-d791b569d671 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:13:25.577031 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:13:25.617006 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b7a5ae-460c-4cca-9139-d791b569d671 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:13:25.622454 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50a209e-16db-4f5c-a6a3-2a7e371791ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b07bf0faca47b28b50990d1720853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:13:25.623056 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50a209e-16db-4f5c-a6a3-2a7e371791ae service nova] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:25.623337 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50a209e-16db-4f5c-a6a3-2a7e371791ae service nova] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:25.623612 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50a209e-16db-4f5c-a6a3-2a7e371791ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:25.631278 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50a209e-16db-4f5c-a6a3-2a7e371791ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:13:25.633349 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50a209e-16db-4f5c-a6a3-2a7e371791ae service nova] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:13:25.642061 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1015/2068] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:13:25 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:13:33.689883 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc9de55-cd56-4efa-a2a6-73dcd3d5ae4a None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:13:33.750336 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc9de55-cd56-4efa-a2a6-73dcd3d5ae4a None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:13:33.754895 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f441b80d-7523-431b-a063-90fae69cecad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:13:33.755514 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f441b80d-7523-431b-a063-90fae69cecad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:33.755791 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f441b80d-7523-431b-a063-90fae69cecad admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:33.756063 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f441b80d-7523-431b-a063-90fae69cecad admin admin] PATH_INFO: `/v3/users/30901704e6a7408cb7fdb8d5479dfd8a` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:33.757205 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f441b80d-7523-431b-a063-90fae69cecad admin admin] RBAC: Authorizing `identity:delete_user(user_id=30901704e6a7408cb7fdb8d5479dfd8a)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:13:33.759184 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f441b80d-7523-431b-a063-90fae69cecad admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:13:33.916841 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-f441b80d-7523-431b-a063-90fae69cecad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30901704e6a7408cb7fdb8d5479dfd8a'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:13:33.917495 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-f441b80d-7523-431b-a063-90fae69cecad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30901704e6a7408cb7fdb8d5479dfd8a'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:13:33.969044 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-f441b80d-7523-431b-a063-90fae69cecad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30901704e6a7408cb7fdb8d5479dfd8a'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:13:33.969656 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-f441b80d-7523-431b-a063-90fae69cecad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30901704e6a7408cb7fdb8d5479dfd8a'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:13:33.971960 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:13:33.978231 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-f441b80d-7523-431b-a063-90fae69cecad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30901704e6a7408cb7fdb8d5479dfd8a'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:13:33.996526 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1054/2069] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:13:33 2022] DELETE /identity/v3/users/30901704e6a7408cb7fdb8d5479dfd8a => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:13:34.001329 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a26f14-4133-48b2-b037-3ef5b258a602 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:34.001329 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a26f14-4133-48b2-b037-3ef5b258a602 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:34.002064 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a26f14-4133-48b2-b037-3ef5b258a602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:34.024070 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-69a26f14-4133-48b2-b037-3ef5b258a602 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:13:34.025273 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-69a26f14-4133-48b2-b037-3ef5b258a602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:13:34.084574 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1016/2070] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:13:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:13:34.095133 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f441b80d-7523-431b-a063-90fae69cecad admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:13:34.096674 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-f441b80d-7523-431b-a063-90fae69cecad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:13:34.155314 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f441b80d-7523-431b-a063-90fae69cecad admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:13:34.158939 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d376001-177d-4e47-9b54-d3fb460b8c3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:13:34.159539 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d376001-177d-4e47-9b54-d3fb460b8c3c service neutron] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:34.159821 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d376001-177d-4e47-9b54-d3fb460b8c3c service neutron] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:34.160095 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d376001-177d-4e47-9b54-d3fb460b8c3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:34.169677 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d376001-177d-4e47-9b54-d3fb460b8c3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:13:34.171091 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d376001-177d-4e47-9b54-d3fb460b8c3c service neutron] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:13:34.180453 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1055/2071] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:13:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:13:34.549889 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a26f14-4133-48b2-b037-3ef5b258a602 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:13:34.552391 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-69a26f14-4133-48b2-b037-3ef5b258a602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:13:34.563417 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a26f14-4133-48b2-b037-3ef5b258a602 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:13:34.568015 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6faaa72-4ee0-4ed1-82cb-65aece0aec2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:13:34.568687 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6faaa72-4ee0-4ed1-82cb-65aece0aec2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:34.568972 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6faaa72-4ee0-4ed1-82cb-65aece0aec2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:34.569253 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6faaa72-4ee0-4ed1-82cb-65aece0aec2a admin admin] PATH_INFO: `/v3/projects/9b612d10ff3b402ea7571f3134c88a11` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:34.570899 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6faaa72-4ee0-4ed1-82cb-65aece0aec2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b612d10ff3b402ea7571f3134c88a11)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:13:34.572830 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6faaa72-4ee0-4ed1-82cb-65aece0aec2a admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:13:34.703293 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-f6faaa72-4ee0-4ed1-82cb-65aece0aec2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b612d10ff3b402ea7571f3134c88a11', 'request_id': 'req-f6faaa72-4ee0-4ed1-82cb-65aece0aec2a'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:13:34.709359 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-f6faaa72-4ee0-4ed1-82cb-65aece0aec2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b612d10ff3b402ea7571f3134c88a11', 'request_id': 'req-f6faaa72-4ee0-4ed1-82cb-65aece0aec2a'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:13:34.782640 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-f6faaa72-4ee0-4ed1-82cb-65aece0aec2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b612d10ff3b402ea7571f3134c88a11', 'request_id': 'req-f6faaa72-4ee0-4ed1-82cb-65aece0aec2a'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:13:34.793776 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-f6faaa72-4ee0-4ed1-82cb-65aece0aec2a admin admin] The token cache is being invalidate because project 9b612d10ff3b402ea7571f3134c88a11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:13:34.794091 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-f6faaa72-4ee0-4ed1-82cb-65aece0aec2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:13:34.794719 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-f6faaa72-4ee0-4ed1-82cb-65aece0aec2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:13:34.796219 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1017/2072] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:13:34 2022] DELETE /identity/v3/projects/9b612d10ff3b402ea7571f3134c88a11 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:13:34.801146 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd85133-6b33-4085-a5f4-e71db2e7a591 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:34.801621 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd85133-6b33-4085-a5f4-e71db2e7a591 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:34.801721 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd85133-6b33-4085-a5f4-e71db2e7a591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:34.822007 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-5dd85133-6b33-4085-a5f4-e71db2e7a591 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:13:34.823196 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-5dd85133-6b33-4085-a5f4-e71db2e7a591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:13:34.880201 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1056/2073] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:13:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:13:34.888528 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bf60cf-a35d-4cab-aa61-18ddf0c687bc None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:34.888813 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bf60cf-a35d-4cab-aa61-18ddf0c687bc None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:34.889087 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bf60cf-a35d-4cab-aa61-18ddf0c687bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:34.908252 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-70bf60cf-a35d-4cab-aa61-18ddf0c687bc None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:13:34.909448 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-70bf60cf-a35d-4cab-aa61-18ddf0c687bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:13:34.917066 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1018/2074] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:13:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:13:34.921088 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd85133-6b33-4085-a5f4-e71db2e7a591 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:13:34.931241 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd85133-6b33-4085-a5f4-e71db2e7a591 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:13:34.935764 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312f2285-e15e-4c65-acc0-6b56fceaada0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:13:34.936393 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312f2285-e15e-4c65-acc0-6b56fceaada0 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:34.936677 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312f2285-e15e-4c65-acc0-6b56fceaada0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:34.936951 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312f2285-e15e-4c65-acc0-6b56fceaada0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:34.937612 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312f2285-e15e-4c65-acc0-6b56fceaada0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 03:13:34.937977 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312f2285-e15e-4c65-acc0-6b56fceaada0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:13:34.939708 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312f2285-e15e-4c65-acc0-6b56fceaada0 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:13:34.951883 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1057/2075] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Oct 14 03:13:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 03:13:34.956377 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bf60cf-a35d-4cab-aa61-18ddf0c687bc None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:13:34.965716 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bf60cf-a35d-4cab-aa61-18ddf0c687bc None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:13:34.970621 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1da2622-2dd4-4bb0-8803-02b4dc9f1ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:13:34.971220 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1da2622-2dd4-4bb0-8803-02b4dc9f1ba0 admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:34.971503 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1da2622-2dd4-4bb0-8803-02b4dc9f1ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:34.971777 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1da2622-2dd4-4bb0-8803-02b4dc9f1ba0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:34.972418 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1da2622-2dd4-4bb0-8803-02b4dc9f1ba0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:13:34.974384 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1da2622-2dd4-4bb0-8803-02b4dc9f1ba0 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:13:35.059201 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1019/2076] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:13:34 2022] POST /identity/v3/projects => generated 373 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:13:35.063303 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312f2285-e15e-4c65-acc0-6b56fceaada0 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:13:35.073316 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312f2285-e15e-4c65-acc0-6b56fceaada0 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:13:35.129350 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f331a9-9c13-4dda-9d06-b732a4f4186e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:13:35.130006 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f331a9-9c13-4dda-9d06-b732a4f4186e admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:35.130286 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f331a9-9c13-4dda-9d06-b732a4f4186e admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:35.130584 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f331a9-9c13-4dda-9d06-b732a4f4186e admin admin] PATH_INFO: `/v3/users` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:35.131246 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f331a9-9c13-4dda-9d06-b732a4f4186e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:13:35.133222 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f331a9-9c13-4dda-9d06-b732a4f4186e admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:13:35.142994 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:13:35.277946 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1058/2077] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:13:35 2022] POST /identity/v3/users => generated 287 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:13:35.282125 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1da2622-2dd4-4bb0-8803-02b4dc9f1ba0 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:13:35.293001 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1da2622-2dd4-4bb0-8803-02b4dc9f1ba0 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:13:35.297991 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3293aaad-b5a2-42a5-87d1-59caf942734f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:13:35.298689 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3293aaad-b5a2-42a5-87d1-59caf942734f admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:35.299298 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3293aaad-b5a2-42a5-87d1-59caf942734f admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:35.299838 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3293aaad-b5a2-42a5-87d1-59caf942734f admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:35.300802 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3293aaad-b5a2-42a5-87d1-59caf942734f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:13:35.302674 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3293aaad-b5a2-42a5-87d1-59caf942734f admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:13:35.305639 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:13:35.314864 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1020/2078] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:13:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 03:13:35.318900 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f331a9-9c13-4dda-9d06-b732a4f4186e admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:13:35.330681 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f331a9-9c13-4dda-9d06-b732a4f4186e admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:13:35.335153 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba4a8a3-aeb5-4c67-abba-1fd5658c0f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:13:35.335823 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba4a8a3-aeb5-4c67-abba-1fd5658c0f01 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:35.336109 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba4a8a3-aeb5-4c67-abba-1fd5658c0f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:35.336504 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba4a8a3-aeb5-4c67-abba-1fd5658c0f01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:35.337156 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba4a8a3-aeb5-4c67-abba-1fd5658c0f01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:13:35.338792 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba4a8a3-aeb5-4c67-abba-1fd5658c0f01 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:13:35.353633 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1059/2079] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:13:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:13:35.354786 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3293aaad-b5a2-42a5-87d1-59caf942734f admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:13:35.364719 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3293aaad-b5a2-42a5-87d1-59caf942734f admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:13:35.369890 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca432b7-9152-4231-96a3-e5a9c76f07c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:13:35.370535 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca432b7-9152-4231-96a3-e5a9c76f07c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:35.370812 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca432b7-9152-4231-96a3-e5a9c76f07c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:35.371087 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca432b7-9152-4231-96a3-e5a9c76f07c2 admin admin] PATH_INFO: `/v3/projects/272c310114fe49558a1ec0d302ce5cfe/users/bad7e2b0895a4c0c8f102ec2edc06c7b/roles/9bd0470b544240209a420f640c79c06c` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:35.386331 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca432b7-9152-4231-96a3-e5a9c76f07c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=272c310114fe49558a1ec0d302ce5cfe, user_id=bad7e2b0895a4c0c8f102ec2edc06c7b, role_id=9bd0470b544240209a420f640c79c06c)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:13:35.388976 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca432b7-9152-4231-96a3-e5a9c76f07c2 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:13:35.494150 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1021/2080] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:13:35 2022] PUT /identity/v3/projects/272c310114fe49558a1ec0d302ce5cfe/users/bad7e2b0895a4c0c8f102ec2edc06c7b/roles/9bd0470b544240209a420f640c79c06c => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 03:13:35.506095 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323b07ad-5f50-484e-b05b-f1c6583b3e9b None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:35.506377 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323b07ad-5f50-484e-b05b-f1c6583b3e9b None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:35.506676 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323b07ad-5f50-484e-b05b-f1c6583b3e9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:35.534223 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-323b07ad-5f50-484e-b05b-f1c6583b3e9b None None] MFA Rules not processed for user `bad7e2b0895a4c0c8f102ec2edc06c7b`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:13:35.535431 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-323b07ad-5f50-484e-b05b-f1c6583b3e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:13:35.598458 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1060/2081] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:13:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 03:13:35.610223 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca432b7-9152-4231-96a3-e5a9c76f07c2 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:13:35.612176 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-aca432b7-9152-4231-96a3-e5a9c76f07c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:13:35.685144 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca432b7-9152-4231-96a3-e5a9c76f07c2 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:13:35.690875 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9101e1d-4520-4fc7-b017-9b200d9aac96 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b07bf0faca47b28b50990d1720853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:13:35.691481 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9101e1d-4520-4fc7-b017-9b200d9aac96 service nova] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:35.691764 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9101e1d-4520-4fc7-b017-9b200d9aac96 service nova] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:35.692037 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9101e1d-4520-4fc7-b017-9b200d9aac96 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:35.699729 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9101e1d-4520-4fc7-b017-9b200d9aac96 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:13:35.701349 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9101e1d-4520-4fc7-b017-9b200d9aac96 service nova] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:13:35.710546 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1022/2082] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:13:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:13:38.841365 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323b07ad-5f50-484e-b05b-f1c6583b3e9b None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:13:38.908143 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323b07ad-5f50-484e-b05b-f1c6583b3e9b None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:13:38.912822 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08162cfa-eb84-4985-ac2f-b31ee2c9c95d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:13:38.913518 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08162cfa-eb84-4985-ac2f-b31ee2c9c95d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:38.913831 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08162cfa-eb84-4985-ac2f-b31ee2c9c95d admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:38.914112 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08162cfa-eb84-4985-ac2f-b31ee2c9c95d admin admin] PATH_INFO: `/v3/users/bad7e2b0895a4c0c8f102ec2edc06c7b` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:38.915816 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08162cfa-eb84-4985-ac2f-b31ee2c9c95d admin admin] RBAC: Authorizing `identity:delete_user(user_id=bad7e2b0895a4c0c8f102ec2edc06c7b)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:13:38.917741 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08162cfa-eb84-4985-ac2f-b31ee2c9c95d admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:13:39.122771 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-08162cfa-eb84-4985-ac2f-b31ee2c9c95d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bad7e2b0895a4c0c8f102ec2edc06c7b'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:13:39.123426 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-08162cfa-eb84-4985-ac2f-b31ee2c9c95d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bad7e2b0895a4c0c8f102ec2edc06c7b'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:13:39.205725 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-08162cfa-eb84-4985-ac2f-b31ee2c9c95d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bad7e2b0895a4c0c8f102ec2edc06c7b'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:13:39.206466 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-08162cfa-eb84-4985-ac2f-b31ee2c9c95d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bad7e2b0895a4c0c8f102ec2edc06c7b'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:13:39.214603 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-08162cfa-eb84-4985-ac2f-b31ee2c9c95d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bad7e2b0895a4c0c8f102ec2edc06c7b'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:13:39.222381 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:INSERT=2,keystone:SELECT=89,keystone:DELETE=9 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:13:39.232351 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1061/2083] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:13:38 2022] DELETE /identity/v3/users/bad7e2b0895a4c0c8f102ec2edc06c7b => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:13:39.237226 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746a8af4-5c57-4de3-8a03-29538a39c01f None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:39.237713 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746a8af4-5c57-4de3-8a03-29538a39c01f None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:39.237832 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746a8af4-5c57-4de3-8a03-29538a39c01f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:39.256710 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-746a8af4-5c57-4de3-8a03-29538a39c01f None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:13:39.257944 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-746a8af4-5c57-4de3-8a03-29538a39c01f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:13:39.315554 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1023/2084] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:13:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:13:39.325815 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08162cfa-eb84-4985-ac2f-b31ee2c9c95d admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:13:39.327387 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-08162cfa-eb84-4985-ac2f-b31ee2c9c95d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:13:39.386819 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08162cfa-eb84-4985-ac2f-b31ee2c9c95d admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:13:39.390494 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb6b63e-7b4a-4d0c-9974-5ec5197723a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:13:39.391094 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb6b63e-7b4a-4d0c-9974-5ec5197723a8 service neutron] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:39.391374 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb6b63e-7b4a-4d0c-9974-5ec5197723a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:39.391645 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb6b63e-7b4a-4d0c-9974-5ec5197723a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:39.401204 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb6b63e-7b4a-4d0c-9974-5ec5197723a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:13:39.402721 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb6b63e-7b4a-4d0c-9974-5ec5197723a8 service neutron] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:13:39.412280 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1062/2085] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:13:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:13:39.813257 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746a8af4-5c57-4de3-8a03-29538a39c01f None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:13:39.814817 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-746a8af4-5c57-4de3-8a03-29538a39c01f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:13:39.831022 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746a8af4-5c57-4de3-8a03-29538a39c01f None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:13:39.836610 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c7fe5d-94ed-4787-871b-e0316429f1e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:13:39.838011 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c7fe5d-94ed-4787-871b-e0316429f1e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:39.838609 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c7fe5d-94ed-4787-871b-e0316429f1e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:39.839104 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c7fe5d-94ed-4787-871b-e0316429f1e3 admin admin] PATH_INFO: `/v3/projects/272c310114fe49558a1ec0d302ce5cfe` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:39.840684 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c7fe5d-94ed-4787-871b-e0316429f1e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=272c310114fe49558a1ec0d302ce5cfe)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:13:39.843349 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c7fe5d-94ed-4787-871b-e0316429f1e3 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:13:39.958002 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-e9c7fe5d-94ed-4787-871b-e0316429f1e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '272c310114fe49558a1ec0d302ce5cfe', 'request_id': 'req-e9c7fe5d-94ed-4787-871b-e0316429f1e3'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:13:39.964741 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-e9c7fe5d-94ed-4787-871b-e0316429f1e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '272c310114fe49558a1ec0d302ce5cfe', 'request_id': 'req-e9c7fe5d-94ed-4787-871b-e0316429f1e3'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:13:39.966407 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:13:40.040769 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-e9c7fe5d-94ed-4787-871b-e0316429f1e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '272c310114fe49558a1ec0d302ce5cfe', 'request_id': 'req-e9c7fe5d-94ed-4787-871b-e0316429f1e3'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:13:40.052763 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-e9c7fe5d-94ed-4787-871b-e0316429f1e3 admin admin] The token cache is being invalidate because project 272c310114fe49558a1ec0d302ce5cfe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:13:40.053213 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-e9c7fe5d-94ed-4787-871b-e0316429f1e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:13:40.054002 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-e9c7fe5d-94ed-4787-871b-e0316429f1e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:13:40.055626 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1024/2086] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:13:39 2022] DELETE /identity/v3/projects/272c310114fe49558a1ec0d302ce5cfe => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:13:40.061198 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce019525-b9ed-437c-90d4-d9fb57726b03 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:40.061677 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce019525-b9ed-437c-90d4-d9fb57726b03 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:40.061928 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce019525-b9ed-437c-90d4-d9fb57726b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:40.082379 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-ce019525-b9ed-437c-90d4-d9fb57726b03 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:13:40.083605 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-ce019525-b9ed-437c-90d4-d9fb57726b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:13:40.145708 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1063/2087] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:13:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:13:40.154065 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a075d3-ad0c-4278-b338-bf982f25ffd2 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:40.155253 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a075d3-ad0c-4278-b338-bf982f25ffd2 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:40.155253 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a075d3-ad0c-4278-b338-bf982f25ffd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:40.175595 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-16a075d3-ad0c-4278-b338-bf982f25ffd2 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:13:40.176991 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-16a075d3-ad0c-4278-b338-bf982f25ffd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:13:40.185707 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1025/2088] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:13:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:13:40.200826 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce019525-b9ed-437c-90d4-d9fb57726b03 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:13:40.200826 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce019525-b9ed-437c-90d4-d9fb57726b03 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:13:40.205425 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848b1578-f1c5-4d16-92d5-badf3ce48ee5 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:40.205788 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848b1578-f1c5-4d16-92d5-badf3ce48ee5 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:40.205896 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568f77d9-ebb3-4921-8f05-4b8219c9ba36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:13:40.206076 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848b1578-f1c5-4d16-92d5-badf3ce48ee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:40.206669 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568f77d9-ebb3-4921-8f05-4b8219c9ba36 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:40.207630 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568f77d9-ebb3-4921-8f05-4b8219c9ba36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:40.207630 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568f77d9-ebb3-4921-8f05-4b8219c9ba36 admin admin] PATH_INFO: `/v3/domains` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:40.208248 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568f77d9-ebb3-4921-8f05-4b8219c9ba36 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 03:13:40.208603 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568f77d9-ebb3-4921-8f05-4b8219c9ba36 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:13:40.210369 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568f77d9-ebb3-4921-8f05-4b8219c9ba36 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:13:40.223698 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1064/2089] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Oct 14 03:13:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 03:13:40.227983 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568f77d9-ebb3-4921-8f05-4b8219c9ba36 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:13:40.228879 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-848b1578-f1c5-4d16-92d5-badf3ce48ee5 None None] MFA Rules not processed for user `44b37050076b42e6af1a67bd7d6d06e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:13:40.230272 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-848b1578-f1c5-4d16-92d5-badf3ce48ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:13:40.239044 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568f77d9-ebb3-4921-8f05-4b8219c9ba36 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:13:40.246577 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c2c0fe-1171-4647-b710-dd53e764fbfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:13:40.247520 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c2c0fe-1171-4647-b710-dd53e764fbfe admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:40.247808 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c2c0fe-1171-4647-b710-dd53e764fbfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:40.248085 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c2c0fe-1171-4647-b710-dd53e764fbfe admin admin] PATH_INFO: `/v3/projects` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:40.248793 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c2c0fe-1171-4647-b710-dd53e764fbfe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:13:40.250825 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c2c0fe-1171-4647-b710-dd53e764fbfe admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:13:40.295548 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1026/2090] 10.222.0.39 () {58 vars in 1009 bytes} [Fri Oct 14 03:13:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:13:40.338964 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1065/2091] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:13:40 2022] POST /identity/v3/projects => generated 367 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:13:40.341631 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848b1578-f1c5-4d16-92d5-badf3ce48ee5 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:13:40.353205 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848b1578-f1c5-4d16-92d5-badf3ce48ee5 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:13:40.408454 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3242acd6-420c-488c-8fd8-709792158acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:13:40.409053 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3242acd6-420c-488c-8fd8-709792158acf admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:40.409336 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3242acd6-420c-488c-8fd8-709792158acf admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:40.409606 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3242acd6-420c-488c-8fd8-709792158acf admin admin] PATH_INFO: `/v3/users` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:40.410261 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3242acd6-420c-488c-8fd8-709792158acf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:13:40.412191 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3242acd6-420c-488c-8fd8-709792158acf admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:13:40.442121 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c2c0fe-1171-4647-b710-dd53e764fbfe admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:13:40.443831 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-86c2c0fe-1171-4647-b710-dd53e764fbfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:13:40.498454 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1027/2092] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:13:40 2022] POST /identity/v3/users => generated 284 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:13:40.499549 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3242acd6-420c-488c-8fd8-709792158acf admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:13:40.514038 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3242acd6-420c-488c-8fd8-709792158acf admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:13:40.518003 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:13:40.518982 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f779f9-42fd-4485-9573-1632571bbf44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:13:40.519568 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f779f9-42fd-4485-9573-1632571bbf44 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:40.519926 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f779f9-42fd-4485-9573-1632571bbf44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:40.520270 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f779f9-42fd-4485-9573-1632571bbf44 admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:40.521035 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f779f9-42fd-4485-9573-1632571bbf44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:13:40.522731 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f779f9-42fd-4485-9573-1632571bbf44 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:13:40.525046 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c2c0fe-1171-4647-b710-dd53e764fbfe admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:13:40.530161 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c2beb8-8016-4daf-8344-fa6604a8d0eb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5e07da64804243bc18862072b027a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:13:40.530898 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c2beb8-8016-4daf-8344-fa6604a8d0eb service swift] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:40.532429 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c2beb8-8016-4daf-8344-fa6604a8d0eb service swift] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:40.532429 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c2beb8-8016-4daf-8344-fa6604a8d0eb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:40.533563 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1028/2093] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:13:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 03:13:40.537223 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f779f9-42fd-4485-9573-1632571bbf44 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:13:40.547150 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f779f9-42fd-4485-9573-1632571bbf44 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:13:40.551863 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca81c1e-af71-4d12-877e-c6dabb1f76cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:13:40.555399 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca81c1e-af71-4d12-877e-c6dabb1f76cc admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:40.555399 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca81c1e-af71-4d12-877e-c6dabb1f76cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:40.555399 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca81c1e-af71-4d12-877e-c6dabb1f76cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:40.555399 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca81c1e-af71-4d12-877e-c6dabb1f76cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:13:40.555399 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca81c1e-af71-4d12-877e-c6dabb1f76cc admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:13:40.566294 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1029/2094] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:13:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 03:13:40.570365 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca81c1e-af71-4d12-877e-c6dabb1f76cc admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:13:40.580781 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca81c1e-af71-4d12-877e-c6dabb1f76cc admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:13:40.585694 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26149e8d-626f-4296-8f69-28dc004c081f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:13:40.586475 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26149e8d-626f-4296-8f69-28dc004c081f admin admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:40.586791 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26149e8d-626f-4296-8f69-28dc004c081f admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:40.587074 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26149e8d-626f-4296-8f69-28dc004c081f admin admin] PATH_INFO: `/v3/projects/74e1e7ae14204c5f9edd7eec9fb54470/users/fa88852f8cbd4f458a8dd0d32ea4d1c7/roles/9bd0470b544240209a420f640c79c06c` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:40.592563 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c2beb8-8016-4daf-8344-fa6604a8d0eb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:13:40.594208 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c2beb8-8016-4daf-8344-fa6604a8d0eb service swift] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:13:40.602238 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26149e8d-626f-4296-8f69-28dc004c081f admin admin] RBAC: Authorizing `identity:create_grant(project_id=74e1e7ae14204c5f9edd7eec9fb54470, user_id=fa88852f8cbd4f458a8dd0d32ea4d1c7, role_id=9bd0470b544240209a420f640c79c06c)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:13:40.602773 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1066/2095] 10.222.0.39 () {60 vars in 1384 bytes} [Fri Oct 14 03:13:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 14 03:13:40.605031 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26149e8d-626f-4296-8f69-28dc004c081f admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:13:40.695585 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1030/2096] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:13:40 2022] PUT /identity/v3/projects/74e1e7ae14204c5f9edd7eec9fb54470/users/fa88852f8cbd4f458a8dd0d32ea4d1c7/roles/9bd0470b544240209a420f640c79c06c => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 03:13:40.706447 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f015daa-7d89-4e46-b56f-f3712c052099 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:40.706738 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f015daa-7d89-4e46-b56f-f3712c052099 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:40.707014 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f015daa-7d89-4e46-b56f-f3712c052099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:40.742583 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-7f015daa-7d89-4e46-b56f-f3712c052099 None None] MFA Rules not processed for user `fa88852f8cbd4f458a8dd0d32ea4d1c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:13:40.744523 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-7f015daa-7d89-4e46-b56f-f3712c052099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:13:40.833931 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1067/2097] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:13:40 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:13:40.845490 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26149e8d-626f-4296-8f69-28dc004c081f admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:13:40.847674 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-26149e8d-626f-4296-8f69-28dc004c081f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:13:40.903662 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:INSERT=5,keystone:DELETE=3,keystone:SELECT=92 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:13:40.945307 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26149e8d-626f-4296-8f69-28dc004c081f admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:13:40.950236 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d28259-b2d6-4f4f-aa3e-2382cf3ca94b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab599d98b0624a2e89090312bcab713f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:13:40.951571 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d28259-b2d6-4f4f-aa3e-2382cf3ca94b service cinder] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:40.951867 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d28259-b2d6-4f4f-aa3e-2382cf3ca94b service cinder] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:40.952152 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d28259-b2d6-4f4f-aa3e-2382cf3ca94b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:40.961787 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d28259-b2d6-4f4f-aa3e-2382cf3ca94b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:13:40.965006 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d28259-b2d6-4f4f-aa3e-2382cf3ca94b service cinder] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:13:40.978426 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1031/2098] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:13:40 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:13:50.829694 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=59 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:13:51.020895 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f015daa-7d89-4e46-b56f-f3712c052099 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:13:51.032163 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=15 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:13:51.113211 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f015daa-7d89-4e46-b56f-f3712c052099 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:13:51.118345 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f084c762-a25d-4bf3-b3ba-02ef7eb54559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:13:51.121906 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f084c762-a25d-4bf3-b3ba-02ef7eb54559 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:51.121906 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f084c762-a25d-4bf3-b3ba-02ef7eb54559 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:51.121906 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f084c762-a25d-4bf3-b3ba-02ef7eb54559 admin admin] PATH_INFO: `/v3/users/fa88852f8cbd4f458a8dd0d32ea4d1c7` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:51.123534 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f084c762-a25d-4bf3-b3ba-02ef7eb54559 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa88852f8cbd4f458a8dd0d32ea4d1c7)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:13:51.125842 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f084c762-a25d-4bf3-b3ba-02ef7eb54559 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:13:51.406365 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-f084c762-a25d-4bf3-b3ba-02ef7eb54559 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa88852f8cbd4f458a8dd0d32ea4d1c7'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:13:51.407032 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-f084c762-a25d-4bf3-b3ba-02ef7eb54559 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa88852f8cbd4f458a8dd0d32ea4d1c7'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:13:51.500226 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-f084c762-a25d-4bf3-b3ba-02ef7eb54559 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa88852f8cbd4f458a8dd0d32ea4d1c7'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:13:51.500836 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-f084c762-a25d-4bf3-b3ba-02ef7eb54559 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa88852f8cbd4f458a8dd0d32ea4d1c7'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:13:51.508987 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-f084c762-a25d-4bf3-b3ba-02ef7eb54559 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa88852f8cbd4f458a8dd0d32ea4d1c7'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:13:51.528290 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1068/2099] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:13:51 2022] DELETE /identity/v3/users/fa88852f8cbd4f458a8dd0d32ea4d1c7 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:13:51.532995 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e12cfdb-bfa0-4eb5-97ff-390eb9a0c94a None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:51.533351 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e12cfdb-bfa0-4eb5-97ff-390eb9a0c94a None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:51.533634 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e12cfdb-bfa0-4eb5-97ff-390eb9a0c94a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:51.554786 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-5e12cfdb-bfa0-4eb5-97ff-390eb9a0c94a None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:13:51.556024 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-5e12cfdb-bfa0-4eb5-97ff-390eb9a0c94a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:13:51.619761 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1032/2100] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:13:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:13:51.629934 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f084c762-a25d-4bf3-b3ba-02ef7eb54559 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:13:51.631625 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-f084c762-a25d-4bf3-b3ba-02ef7eb54559 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:13:51.689951 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f084c762-a25d-4bf3-b3ba-02ef7eb54559 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:13:51.693455 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb3f874-0b7b-4611-bc2f-4a9cc83ecb4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:13:51.694084 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb3f874-0b7b-4611-bc2f-4a9cc83ecb4f service neutron] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:51.694366 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb3f874-0b7b-4611-bc2f-4a9cc83ecb4f service neutron] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:51.694633 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb3f874-0b7b-4611-bc2f-4a9cc83ecb4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:51.703734 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb3f874-0b7b-4611-bc2f-4a9cc83ecb4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:13:51.705197 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb3f874-0b7b-4611-bc2f-4a9cc83ecb4f service neutron] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:13:51.715211 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1069/2101] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:13:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:13:52.084812 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e12cfdb-bfa0-4eb5-97ff-390eb9a0c94a None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:13:52.086410 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-5e12cfdb-bfa0-4eb5-97ff-390eb9a0c94a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:13:52.099287 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e12cfdb-bfa0-4eb5-97ff-390eb9a0c94a None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:13:52.104140 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a12551-ebd4-459f-9eea-d81a263c890c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:13:52.104779 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a12551-ebd4-459f-9eea-d81a263c890c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:52.105063 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a12551-ebd4-459f-9eea-d81a263c890c admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:52.108395 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a12551-ebd4-459f-9eea-d81a263c890c admin admin] PATH_INFO: `/v3/projects/74e1e7ae14204c5f9edd7eec9fb54470` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:52.108395 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a12551-ebd4-459f-9eea-d81a263c890c admin admin] RBAC: Authorizing `identity:delete_project(project_id=74e1e7ae14204c5f9edd7eec9fb54470)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:13:52.108587 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a12551-ebd4-459f-9eea-d81a263c890c admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:13:52.206059 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-80a12551-ebd4-459f-9eea-d81a263c890c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74e1e7ae14204c5f9edd7eec9fb54470', 'request_id': 'req-80a12551-ebd4-459f-9eea-d81a263c890c'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:13:52.212180 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-80a12551-ebd4-459f-9eea-d81a263c890c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74e1e7ae14204c5f9edd7eec9fb54470', 'request_id': 'req-80a12551-ebd4-459f-9eea-d81a263c890c'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:13:52.271785 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-80a12551-ebd4-459f-9eea-d81a263c890c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74e1e7ae14204c5f9edd7eec9fb54470', 'request_id': 'req-80a12551-ebd4-459f-9eea-d81a263c890c'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:13:52.284126 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-80a12551-ebd4-459f-9eea-d81a263c890c admin admin] The token cache is being invalidate because project 74e1e7ae14204c5f9edd7eec9fb54470 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:13:52.284425 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-80a12551-ebd4-459f-9eea-d81a263c890c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:13:52.285080 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-80a12551-ebd4-459f-9eea-d81a263c890c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:13:52.286676 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1033/2102] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:13:52 2022] DELETE /identity/v3/projects/74e1e7ae14204c5f9edd7eec9fb54470 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:13:52.291682 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd6a979-d139-405d-a234-c4d7feb793b3 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:52.292259 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd6a979-d139-405d-a234-c4d7feb793b3 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:52.292358 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd6a979-d139-405d-a234-c4d7feb793b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:52.312848 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-afd6a979-d139-405d-a234-c4d7feb793b3 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:13:52.314198 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-afd6a979-d139-405d-a234-c4d7feb793b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:13:52.332667 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:13:52.385045 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1070/2103] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:13:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:13:52.390300 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171b3095-5651-4fe3-83da-0933f89bae45 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:52.391443 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171b3095-5651-4fe3-83da-0933f89bae45 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:52.391443 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171b3095-5651-4fe3-83da-0933f89bae45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:52.413189 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-171b3095-5651-4fe3-83da-0933f89bae45 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:13:52.414522 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-171b3095-5651-4fe3-83da-0933f89bae45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:13:52.423173 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1034/2104] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:13:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:13:52.432965 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd6a979-d139-405d-a234-c4d7feb793b3 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:13:52.442447 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd6a979-d139-405d-a234-c4d7feb793b3 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:13:52.448232 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0403c4-aa0a-4388-8a1c-4a28e4e6bb7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:13:52.448832 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0403c4-aa0a-4388-8a1c-4a28e4e6bb7c admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:52.449113 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0403c4-aa0a-4388-8a1c-4a28e4e6bb7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:52.449386 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0403c4-aa0a-4388-8a1c-4a28e4e6bb7c admin admin] PATH_INFO: `/v3/domains` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:52.450070 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0403c4-aa0a-4388-8a1c-4a28e4e6bb7c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 03:13:52.450378 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0403c4-aa0a-4388-8a1c-4a28e4e6bb7c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:13:52.451996 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0403c4-aa0a-4388-8a1c-4a28e4e6bb7c admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:13:52.463724 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1071/2105] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Oct 14 03:13:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 03:13:52.468034 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171b3095-5651-4fe3-83da-0933f89bae45 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:13:52.477335 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171b3095-5651-4fe3-83da-0933f89bae45 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:13:52.482212 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7537e32-9f78-4e6c-a783-08825af013d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:13:52.482817 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7537e32-9f78-4e6c-a783-08825af013d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:52.483099 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7537e32-9f78-4e6c-a783-08825af013d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:52.483374 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7537e32-9f78-4e6c-a783-08825af013d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:52.484011 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7537e32-9f78-4e6c-a783-08825af013d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:13:52.486320 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7537e32-9f78-4e6c-a783-08825af013d4 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:13:52.625919 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1035/2106] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:13:52 2022] POST /identity/v3/projects => generated 371 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:13:52.629441 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0403c4-aa0a-4388-8a1c-4a28e4e6bb7c admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:13:52.640791 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0403c4-aa0a-4388-8a1c-4a28e4e6bb7c admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:13:52.724943 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68a445d-1d69-48c2-9af1-18b930f64b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:13:52.725822 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68a445d-1d69-48c2-9af1-18b930f64b73 admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:52.726208 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68a445d-1d69-48c2-9af1-18b930f64b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:52.726552 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68a445d-1d69-48c2-9af1-18b930f64b73 admin admin] PATH_INFO: `/v3/users` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:52.727364 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68a445d-1d69-48c2-9af1-18b930f64b73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:13:52.730237 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68a445d-1d69-48c2-9af1-18b930f64b73 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:13:52.849088 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1072/2107] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:13:52 2022] POST /identity/v3/users => generated 286 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:13:52.853607 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7537e32-9f78-4e6c-a783-08825af013d4 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:13:52.864425 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7537e32-9f78-4e6c-a783-08825af013d4 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:13:52.869843 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070494c6-9aed-4ad3-be0b-b381124c6604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:13:52.870392 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070494c6-9aed-4ad3-be0b-b381124c6604 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:52.870701 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070494c6-9aed-4ad3-be0b-b381124c6604 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:52.870972 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070494c6-9aed-4ad3-be0b-b381124c6604 admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:52.871617 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070494c6-9aed-4ad3-be0b-b381124c6604 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:13:52.873219 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070494c6-9aed-4ad3-be0b-b381124c6604 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:13:52.883237 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1036/2108] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:13:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:13:52.887261 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68a445d-1d69-48c2-9af1-18b930f64b73 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:13:52.897334 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68a445d-1d69-48c2-9af1-18b930f64b73 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:13:52.902096 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047dac8c-bb65-4ae4-ae45-fa1a8b466756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:13:52.902688 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047dac8c-bb65-4ae4-ae45-fa1a8b466756 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:52.902968 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047dac8c-bb65-4ae4-ae45-fa1a8b466756 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:52.903240 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047dac8c-bb65-4ae4-ae45-fa1a8b466756 admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:52.903876 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047dac8c-bb65-4ae4-ae45-fa1a8b466756 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:13:52.905513 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047dac8c-bb65-4ae4-ae45-fa1a8b466756 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:13:52.916509 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1073/2109] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:13:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:13:52.920477 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070494c6-9aed-4ad3-be0b-b381124c6604 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:13:52.930314 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070494c6-9aed-4ad3-be0b-b381124c6604 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:13:52.935421 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f3f522-d839-4ba2-b940-2d5cf120391f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:13:52.936055 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f3f522-d839-4ba2-b940-2d5cf120391f admin admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:52.936348 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f3f522-d839-4ba2-b940-2d5cf120391f admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:52.936616 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f3f522-d839-4ba2-b940-2d5cf120391f admin admin] PATH_INFO: `/v3/projects/fb4f5c08cdc64a289362d797e8eb9c64/users/2280c3861ca4497985d0b34071974b22/roles/9bd0470b544240209a420f640c79c06c` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:52.950824 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f3f522-d839-4ba2-b940-2d5cf120391f admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb4f5c08cdc64a289362d797e8eb9c64, user_id=2280c3861ca4497985d0b34071974b22, role_id=9bd0470b544240209a420f640c79c06c)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:13:52.953467 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f3f522-d839-4ba2-b940-2d5cf120391f admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:13:53.065880 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1037/2110] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:13:52 2022] PUT /identity/v3/projects/fb4f5c08cdc64a289362d797e8eb9c64/users/2280c3861ca4497985d0b34071974b22/roles/9bd0470b544240209a420f640c79c06c => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:13:53.076455 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c5046-4848-46d7-bbb2-111d37fca4de None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:53.076741 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c5046-4848-46d7-bbb2-111d37fca4de None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:53.077019 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c5046-4848-46d7-bbb2-111d37fca4de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:53.105898 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-bd1c5046-4848-46d7-bbb2-111d37fca4de None None] MFA Rules not processed for user `2280c3861ca4497985d0b34071974b22`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:13:53.107090 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-bd1c5046-4848-46d7-bbb2-111d37fca4de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:13:53.169427 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1074/2111] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:13:53 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 03:13:53.173926 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f3f522-d839-4ba2-b940-2d5cf120391f admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:13:53.194870 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:13:53.235523 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f3f522-d839-4ba2-b940-2d5cf120391f admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:13:53.240667 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5877a6a-7aa2-4ae5-b389-4109c5f584e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:13:53.241300 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5877a6a-7aa2-4ae5-b389-4109c5f584e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:53.241581 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5877a6a-7aa2-4ae5-b389-4109c5f584e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:53.241887 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5877a6a-7aa2-4ae5-b389-4109c5f584e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:53.242524 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5877a6a-7aa2-4ae5-b389-4109c5f584e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:13:53.244530 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5877a6a-7aa2-4ae5-b389-4109c5f584e9 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:13:53.341442 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1038/2112] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:13:53 2022] POST /identity/v3/projects => generated 373 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:13:53.345515 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1c5046-4848-46d7-bbb2-111d37fca4de None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:13:53.356837 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1c5046-4848-46d7-bbb2-111d37fca4de None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:13:53.369184 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:13:53.411919 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4d564d-5a46-43c7-bbd8-1a1981f98f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:13:53.412518 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4d564d-5a46-43c7-bbd8-1a1981f98f68 admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:53.412799 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4d564d-5a46-43c7-bbd8-1a1981f98f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:53.413076 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4d564d-5a46-43c7-bbd8-1a1981f98f68 admin admin] PATH_INFO: `/v3/users` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:53.413699 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4d564d-5a46-43c7-bbd8-1a1981f98f68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:13:53.416344 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4d564d-5a46-43c7-bbd8-1a1981f98f68 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:13:53.486242 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1075/2113] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:13:53 2022] POST /identity/v3/users => generated 293 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:13:53.490200 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5877a6a-7aa2-4ae5-b389-4109c5f584e9 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:13:53.500564 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5877a6a-7aa2-4ae5-b389-4109c5f584e9 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:13:53.505333 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464bcf0e-284b-4fed-8644-ba879cf2ea7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:13:53.505961 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464bcf0e-284b-4fed-8644-ba879cf2ea7a admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:53.506241 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464bcf0e-284b-4fed-8644-ba879cf2ea7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:53.506510 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464bcf0e-284b-4fed-8644-ba879cf2ea7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:53.507297 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464bcf0e-284b-4fed-8644-ba879cf2ea7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:13:53.508831 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464bcf0e-284b-4fed-8644-ba879cf2ea7a admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:13:53.523361 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1039/2114] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:13:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 03:13:53.524532 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4d564d-5a46-43c7-bbd8-1a1981f98f68 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:13:53.535628 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4d564d-5a46-43c7-bbd8-1a1981f98f68 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:13:53.540868 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85461937-df85-4fab-940e-741212a668f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:13:53.541694 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85461937-df85-4fab-940e-741212a668f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:53.542013 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85461937-df85-4fab-940e-741212a668f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:53.542288 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85461937-df85-4fab-940e-741212a668f2 admin admin] PATH_INFO: `/v3/projects/a4534457a4d64a79b0cd39461409d95a/users/c3c9d7d1dd1c450b836e39bc4face951/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:53.557715 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85461937-df85-4fab-940e-741212a668f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4534457a4d64a79b0cd39461409d95a, user_id=c3c9d7d1dd1c450b836e39bc4face951, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:13:53.560609 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85461937-df85-4fab-940e-741212a668f2 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:13:53.628704 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1076/2115] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:13:53 2022] PUT /identity/v3/projects/a4534457a4d64a79b0cd39461409d95a/users/c3c9d7d1dd1c450b836e39bc4face951/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:13:53.632562 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464bcf0e-284b-4fed-8644-ba879cf2ea7a admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:13:53.643038 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464bcf0e-284b-4fed-8644-ba879cf2ea7a admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:13:53.698982 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426edd77-eb5e-4ad9-b4e9-4b5ff8285a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:13:53.699582 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426edd77-eb5e-4ad9-b4e9-4b5ff8285a8d admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:53.699866 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426edd77-eb5e-4ad9-b4e9-4b5ff8285a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:53.700138 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426edd77-eb5e-4ad9-b4e9-4b5ff8285a8d admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:53.700775 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426edd77-eb5e-4ad9-b4e9-4b5ff8285a8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:13:53.702400 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426edd77-eb5e-4ad9-b4e9-4b5ff8285a8d admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:13:53.712095 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1040/2116] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:13:53 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:13:53.723383 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85461937-df85-4fab-940e-741212a668f2 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:13:53.728166 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85461937-df85-4fab-940e-741212a668f2 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:13:53.732856 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c16fb0f-245b-4557-aef6-6925a114682a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:13:53.733512 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c16fb0f-245b-4557-aef6-6925a114682a admin admin] REQUEST_METHOD: `PUT` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:53.733859 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c16fb0f-245b-4557-aef6-6925a114682a admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:53.734145 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c16fb0f-245b-4557-aef6-6925a114682a admin admin] PATH_INFO: `/v3/projects/a4534457a4d64a79b0cd39461409d95a/users/c3c9d7d1dd1c450b836e39bc4face951/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:53.736404 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c16fb0f-245b-4557-aef6-6925a114682a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4534457a4d64a79b0cd39461409d95a, user_id=c3c9d7d1dd1c450b836e39bc4face951, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:13:53.739152 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c16fb0f-245b-4557-aef6-6925a114682a admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:13:53.749321 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1077/2117] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:13:53 2022] PUT /identity/v3/projects/a4534457a4d64a79b0cd39461409d95a/users/c3c9d7d1dd1c450b836e39bc4face951/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:13:53.761117 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc9c64-3f45-472b-a4d9-ac032432300d None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:53.761406 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc9c64-3f45-472b-a4d9-ac032432300d None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:53.761686 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc9c64-3f45-472b-a4d9-ac032432300d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:53.789366 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-5ddc9c64-3f45-472b-a4d9-ac032432300d None None] MFA Rules not processed for user `c3c9d7d1dd1c450b836e39bc4face951`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:13:53.790773 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-5ddc9c64-3f45-472b-a4d9-ac032432300d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:13:53.856249 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1041/2118] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:13:53 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:13:53.867633 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c16fb0f-245b-4557-aef6-6925a114682a admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:13:53.869878 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-4c16fb0f-245b-4557-aef6-6925a114682a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:13:53.926100 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c16fb0f-245b-4557-aef6-6925a114682a admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:13:53.929636 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40128518-3f3f-43da-90ef-f4f4c75016ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab599d98b0624a2e89090312bcab713f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:13:53.930218 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40128518-3f3f-43da-90ef-f4f4c75016ce service cinder] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:53.930500 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40128518-3f3f-43da-90ef-f4f4c75016ce service cinder] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:53.930777 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40128518-3f3f-43da-90ef-f4f4c75016ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:53.940321 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40128518-3f3f-43da-90ef-f4f4c75016ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:13:53.941739 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40128518-3f3f-43da-90ef-f4f4c75016ce service cinder] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:13:53.951548 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1078/2119] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:13:53 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:13:54.150607 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddc9c64-3f45-472b-a4d9-ac032432300d None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:13:54.160486 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddc9c64-3f45-472b-a4d9-ac032432300d None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:13:54.164175 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bea618-17a6-476c-9238-cec09f4c39ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab599d98b0624a2e89090312bcab713f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:13:54.164759 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bea618-17a6-476c-9238-cec09f4c39ce service cinder] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:54.165037 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bea618-17a6-476c-9238-cec09f4c39ce service cinder] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:54.165307 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bea618-17a6-476c-9238-cec09f4c39ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:54.177318 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:13:54.221701 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bea618-17a6-476c-9238-cec09f4c39ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:13:54.223099 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bea618-17a6-476c-9238-cec09f4c39ce service cinder] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:13:54.232865 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1042/2120] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:13:54 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:13:58.585447 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6ecb11-68f0-4642-9f88-402815c20f8e None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:58.587076 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6ecb11-68f0-4642-9f88-402815c20f8e None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:58.587076 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6ecb11-68f0-4642-9f88-402815c20f8e None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:58.587476 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1079/2121] 10.222.0.39 () {54 vars in 921 bytes} [Fri Oct 14 03:13:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 03:13:58.593491 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeaf6c8-a741-4249-a8af-6cd79730d91d None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:58.593994 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeaf6c8-a741-4249-a8af-6cd79730d91d None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:58.594090 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeaf6c8-a741-4249-a8af-6cd79730d91d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:58.618079 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-cfeaf6c8-a741-4249-a8af-6cd79730d91d None None] MFA Rules not processed for user `a6b07bf0faca47b28b50990d1720853f`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:13:58.619497 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-cfeaf6c8-a741-4249-a8af-6cd79730d91d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:13:58.705082 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1043/2122] 10.222.0.39 () {58 vars in 1018 bytes} [Fri Oct 14 03:13:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:13:58.716812 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6ecb11-68f0-4642-9f88-402815c20f8e None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:13:58.718515 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-4d6ecb11-68f0-4642-9f88-402815c20f8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:13:58.778722 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6ecb11-68f0-4642-9f88-402815c20f8e None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:13:58.782291 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f649275-5534-4988-a90c-69f7bc9e3199 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a3cd1fbe6df48f8aa90d4ea69905e9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:13:58.782891 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f649275-5534-4988-a90c-69f7bc9e3199 service glance] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:58.783174 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f649275-5534-4988-a90c-69f7bc9e3199 service glance] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:58.783450 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f649275-5534-4988-a90c-69f7bc9e3199 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:58.794116 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f649275-5534-4988-a90c-69f7bc9e3199 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:13:58.795573 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f649275-5534-4988-a90c-69f7bc9e3199 service glance] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:13:58.806296 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1080/2123] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:13:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:13:59.064197 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15685e4e-2e59-40f7-b27b-ca677263cf87 None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:59.064488 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15685e4e-2e59-40f7-b27b-ca677263cf87 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:59.064784 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15685e4e-2e59-40f7-b27b-ca677263cf87 None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:59.066220 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1044/2124] 10.222.0.39 () {54 vars in 921 bytes} [Fri Oct 14 03:13:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 03:13:59.070944 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d14e52f-6da6-4386-87e7-0e157d66bfd2 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:59.071232 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d14e52f-6da6-4386-87e7-0e157d66bfd2 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:59.071512 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d14e52f-6da6-4386-87e7-0e157d66bfd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:59.091262 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-7d14e52f-6da6-4386-87e7-0e157d66bfd2 None None] MFA Rules not processed for user `b123c412ac714b06929a4bd86eca71e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:13:59.092494 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-7d14e52f-6da6-4386-87e7-0e157d66bfd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:13:59.102544 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:13:59.143614 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1081/2125] 10.222.0.39 () {58 vars in 1018 bytes} [Fri Oct 14 03:13:59 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:13:59.156254 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15685e4e-2e59-40f7-b27b-ca677263cf87 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:13:59.158018 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-15685e4e-2e59-40f7-b27b-ca677263cf87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:13:59.171814 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15685e4e-2e59-40f7-b27b-ca677263cf87 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:13:59.175438 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da22dd7-74db-4e17-9b1f-6ce8b568b3a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:13:59.177294 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da22dd7-74db-4e17-9b1f-6ce8b568b3a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:59.177294 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da22dd7-74db-4e17-9b1f-6ce8b568b3a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:59.177294 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da22dd7-74db-4e17-9b1f-6ce8b568b3a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:59.184134 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da22dd7-74db-4e17-9b1f-6ce8b568b3a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:13:59.185632 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da22dd7-74db-4e17-9b1f-6ce8b568b3a3 service neutron] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:13:59.194141 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1045/2126] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:13:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:13:59.282564 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e980f241-7dc7-4568-ac32-13b2ca284321 None None] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:59.282885 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e980f241-7dc7-4568-ac32-13b2ca284321 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:59.283177 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e980f241-7dc7-4568-ac32-13b2ca284321 None None] PATH_INFO: `/` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:59.284533 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1082/2127] 10.222.0.39 () {54 vars in 921 bytes} [Fri Oct 14 03:13:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 03:13:59.289174 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f47e1ae-ac5e-409d-86bd-5c535e48435e None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:59.289467 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f47e1ae-ac5e-409d-86bd-5c535e48435e None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:59.289744 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f47e1ae-ac5e-409d-86bd-5c535e48435e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:59.309881 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-8f47e1ae-ac5e-409d-86bd-5c535e48435e None None] MFA Rules not processed for user `a6b07bf0faca47b28b50990d1720853f`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:13:59.311338 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-8f47e1ae-ac5e-409d-86bd-5c535e48435e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:13:59.320724 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1046/2128] 10.222.0.39 () {58 vars in 1018 bytes} [Fri Oct 14 03:13:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:13:59.331500 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e980f241-7dc7-4568-ac32-13b2ca284321 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:13:59.333173 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-e980f241-7dc7-4568-ac32-13b2ca284321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:13:59.346843 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e980f241-7dc7-4568-ac32-13b2ca284321 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:13:59.350437 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9ce059-2547-4acb-8fe0-650354f744bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:13:59.351024 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ce059-2547-4acb-8fe0-650354f744bf service neutron] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:13:59.351306 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ce059-2547-4acb-8fe0-650354f744bf service neutron] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:13:59.351577 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ce059-2547-4acb-8fe0-650354f744bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:13:59.362352 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9ce059-2547-4acb-8fe0-650354f744bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:13:59.363761 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9ce059-2547-4acb-8fe0-650354f744bf service neutron] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:13:59.374310 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1083/2129] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:13:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:14:03.141679 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7919b5b3-7962-4d0e-8a62-5210c77b172b None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:14:03.142184 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7919b5b3-7962-4d0e-8a62-5210c77b172b None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:14:03.142286 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7919b5b3-7962-4d0e-8a62-5210c77b172b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:14:03.181286 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-7919b5b3-7962-4d0e-8a62-5210c77b172b None None] MFA Rules not processed for user `44b37050076b42e6af1a67bd7d6d06e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:14:03.183156 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-7919b5b3-7962-4d0e-8a62-5210c77b172b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:14:03.253827 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:14:03.267467 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1047/2130] 10.222.0.39 () {58 vars in 1009 bytes} [Fri Oct 14 03:14:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:14:03.279414 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9ce059-2547-4acb-8fe0-650354f744bf service neutron] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:14:03.280809 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-ca9ce059-2547-4acb-8fe0-650354f744bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:14:03.417343 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9ce059-2547-4acb-8fe0-650354f744bf service neutron] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:14:03.422736 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e965c8-24bd-427f-b01e-0a461ddee801 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5e07da64804243bc18862072b027a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:14:03.423429 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e965c8-24bd-427f-b01e-0a461ddee801 service swift] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:14:03.423742 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e965c8-24bd-427f-b01e-0a461ddee801 service swift] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:14:03.424019 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e965c8-24bd-427f-b01e-0a461ddee801 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:14:03.433171 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e965c8-24bd-427f-b01e-0a461ddee801 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:14:03.434795 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e965c8-24bd-427f-b01e-0a461ddee801 service swift] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:14:03.443311 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1084/2131] 10.222.0.39 () {60 vars in 1384 bytes} [Fri Oct 14 03:14:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 14 03:14:13.359372 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=3 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:14:13.439657 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=68 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:14:19.822750 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370a6fe5-2de5-467d-9349-6e85efc95372 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:14:19.823363 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370a6fe5-2de5-467d-9349-6e85efc95372 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:14:19.823363 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370a6fe5-2de5-467d-9349-6e85efc95372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:14:19.842831 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-370a6fe5-2de5-467d-9349-6e85efc95372 None None] MFA Rules not processed for user `44b37050076b42e6af1a67bd7d6d06e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:14:19.844026 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-370a6fe5-2de5-467d-9349-6e85efc95372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:14:19.851135 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1048/2132] 10.222.0.39 () {58 vars in 967 bytes} [Fri Oct 14 03:14:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:14:19.866911 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e965c8-24bd-427f-b01e-0a461ddee801 service swift] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:14:19.875960 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e965c8-24bd-427f-b01e-0a461ddee801 service swift] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:14:19.880848 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba76503-d3cb-441f-b0e4-b37821cbf388 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5e07da64804243bc18862072b027a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:14:19.881430 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba76503-d3cb-441f-b0e4-b37821cbf388 service swift] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:14:19.881711 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba76503-d3cb-441f-b0e4-b37821cbf388 service swift] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:14:19.882030 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba76503-d3cb-441f-b0e4-b37821cbf388 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:14:19.889857 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba76503-d3cb-441f-b0e4-b37821cbf388 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:14:19.891419 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba76503-d3cb-441f-b0e4-b37821cbf388 service swift] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:14:19.899451 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1085/2133] 10.222.0.39 () {60 vars in 1384 bytes} [Fri Oct 14 03:14:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 14 03:14:29.838007 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=3 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:14:29.896288 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=6 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:14:43.703874 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370a6fe5-2de5-467d-9349-6e85efc95372 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:14:43.723286 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370a6fe5-2de5-467d-9349-6e85efc95372 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:14:43.727002 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c467cc89-02cd-4970-b4af-ba286136134a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:14:43.727620 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c467cc89-02cd-4970-b4af-ba286136134a service neutron] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:14:43.727899 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c467cc89-02cd-4970-b4af-ba286136134a service neutron] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:14:43.731138 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c467cc89-02cd-4970-b4af-ba286136134a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:14:43.731138 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-c467cc89-02cd-4970-b4af-ba286136134a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:14:43.819782 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c467cc89-02cd-4970-b4af-ba286136134a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:14:43.821233 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c467cc89-02cd-4970-b4af-ba286136134a service neutron] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:14:43.840212 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1049/2134] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:14:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:14:45.598924 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba76503-d3cb-441f-b0e4-b37821cbf388 service swift] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:14:45.612857 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba76503-d3cb-441f-b0e4-b37821cbf388 service swift] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:14:45.618878 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfad8cc3-ee64-44d2-b9b8-6d7f9a6f783f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:14:45.620079 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfad8cc3-ee64-44d2-b9b8-6d7f9a6f783f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:14:45.620568 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfad8cc3-ee64-44d2-b9b8-6d7f9a6f783f admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:14:45.621000 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfad8cc3-ee64-44d2-b9b8-6d7f9a6f783f admin admin] PATH_INFO: `/v3/users/2280c3861ca4497985d0b34071974b22` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:14:45.623557 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfad8cc3-ee64-44d2-b9b8-6d7f9a6f783f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2280c3861ca4497985d0b34071974b22)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:14:45.626188 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfad8cc3-ee64-44d2-b9b8-6d7f9a6f783f admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:14:45.833883 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c467cc89-02cd-4970-b4af-ba286136134a service neutron] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:14:45.848319 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c467cc89-02cd-4970-b4af-ba286136134a service neutron] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:14:45.852790 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5c7527-8877-4131-ae0d-33375f18257e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:14:45.853859 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c7527-8877-4131-ae0d-33375f18257e service neutron] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:14:45.854301 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c7527-8877-4131-ae0d-33375f18257e service neutron] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:14:45.855078 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c7527-8877-4131-ae0d-33375f18257e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:14:45.858012 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-cc5c7527-8877-4131-ae0d-33375f18257e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:14:45.861399 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-cfad8cc3-ee64-44d2-b9b8-6d7f9a6f783f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2280c3861ca4497985d0b34071974b22'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:14:45.862944 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-cfad8cc3-ee64-44d2-b9b8-6d7f9a6f783f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2280c3861ca4497985d0b34071974b22'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:14:45.877619 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5c7527-8877-4131-ae0d-33375f18257e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:14:45.879543 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5c7527-8877-4131-ae0d-33375f18257e service neutron] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:14:45.891903 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1050/2135] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:14:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 03:14:45.937792 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-cfad8cc3-ee64-44d2-b9b8-6d7f9a6f783f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2280c3861ca4497985d0b34071974b22'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:14:45.938838 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-cfad8cc3-ee64-44d2-b9b8-6d7f9a6f783f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2280c3861ca4497985d0b34071974b22'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:14:45.950824 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-cfad8cc3-ee64-44d2-b9b8-6d7f9a6f783f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2280c3861ca4497985d0b34071974b22'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:14:45.995874 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1086/2136] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:14:45 2022] DELETE /identity/v3/users/2280c3861ca4497985d0b34071974b22 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:14:46.000914 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f835fefb-9e7e-4899-8e96-ab6cff4d836f None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:14:46.002750 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f835fefb-9e7e-4899-8e96-ab6cff4d836f None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:14:46.003222 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f835fefb-9e7e-4899-8e96-ab6cff4d836f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:14:46.061188 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-f835fefb-9e7e-4899-8e96-ab6cff4d836f None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:14:46.062709 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-f835fefb-9e7e-4899-8e96-ab6cff4d836f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:14:46.150761 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1051/2137] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:14:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 03:14:46.177885 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfad8cc3-ee64-44d2-b9b8-6d7f9a6f783f admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:14:46.179437 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-cfad8cc3-ee64-44d2-b9b8-6d7f9a6f783f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:14:46.292283 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfad8cc3-ee64-44d2-b9b8-6d7f9a6f783f admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:14:46.296422 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3392727-f651-47bf-b30d-84b675537451 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:14:46.297030 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3392727-f651-47bf-b30d-84b675537451 service neutron] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:14:46.297310 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3392727-f651-47bf-b30d-84b675537451 service neutron] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:14:46.297581 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3392727-f651-47bf-b30d-84b675537451 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:14:46.307375 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3392727-f651-47bf-b30d-84b675537451 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:14:46.308811 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3392727-f651-47bf-b30d-84b675537451 service neutron] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:14:46.318736 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1087/2138] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:14:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:14:47.101238 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f835fefb-9e7e-4899-8e96-ab6cff4d836f None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:14:47.106610 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-f835fefb-9e7e-4899-8e96-ab6cff4d836f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:14:47.142688 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f835fefb-9e7e-4899-8e96-ab6cff4d836f None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:14:47.154538 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1b336f-1a64-435b-b0cf-b9acb49cc4b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:14:47.155199 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b336f-1a64-435b-b0cf-b9acb49cc4b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:14:47.160060 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b336f-1a64-435b-b0cf-b9acb49cc4b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:14:47.160060 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b336f-1a64-435b-b0cf-b9acb49cc4b9 admin admin] PATH_INFO: `/v3/projects/fb4f5c08cdc64a289362d797e8eb9c64` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:14:47.160060 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1b336f-1a64-435b-b0cf-b9acb49cc4b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb4f5c08cdc64a289362d797e8eb9c64)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:14:47.160060 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1b336f-1a64-435b-b0cf-b9acb49cc4b9 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:14:47.309565 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:14:47.336261 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-ae1b336f-1a64-435b-b0cf-b9acb49cc4b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb4f5c08cdc64a289362d797e8eb9c64', 'request_id': 'req-ae1b336f-1a64-435b-b0cf-b9acb49cc4b9'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:14:47.349210 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-ae1b336f-1a64-435b-b0cf-b9acb49cc4b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb4f5c08cdc64a289362d797e8eb9c64', 'request_id': 'req-ae1b336f-1a64-435b-b0cf-b9acb49cc4b9'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:14:47.420449 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-ae1b336f-1a64-435b-b0cf-b9acb49cc4b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb4f5c08cdc64a289362d797e8eb9c64', 'request_id': 'req-ae1b336f-1a64-435b-b0cf-b9acb49cc4b9'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:14:47.431431 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-ae1b336f-1a64-435b-b0cf-b9acb49cc4b9 admin admin] The token cache is being invalidate because project fb4f5c08cdc64a289362d797e8eb9c64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:14:47.431723 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-ae1b336f-1a64-435b-b0cf-b9acb49cc4b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:14:47.440564 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-ae1b336f-1a64-435b-b0cf-b9acb49cc4b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:14:47.440564 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1052/2139] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:14:47 2022] DELETE /identity/v3/projects/fb4f5c08cdc64a289362d797e8eb9c64 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:14:47.441018 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3392727-f651-47bf-b30d-84b675537451 service neutron] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:14:47.450935 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3392727-f651-47bf-b30d-84b675537451 service neutron] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:14:47.516811 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163f3df4-9878-42e9-a3d6-d025d351016a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:14:47.518077 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163f3df4-9878-42e9-a3d6-d025d351016a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:14:47.518616 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163f3df4-9878-42e9-a3d6-d025d351016a admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:14:47.519219 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163f3df4-9878-42e9-a3d6-d025d351016a admin admin] PATH_INFO: `/v3/users/c3c9d7d1dd1c450b836e39bc4face951` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:14:47.520697 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163f3df4-9878-42e9-a3d6-d025d351016a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3c9d7d1dd1c450b836e39bc4face951)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:14:47.524074 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163f3df4-9878-42e9-a3d6-d025d351016a admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:14:47.538821 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:14:47.722205 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1b336f-1a64-435b-b0cf-b9acb49cc4b9 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:14:47.724126 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-ae1b336f-1a64-435b-b0cf-b9acb49cc4b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:14:47.790905 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-163f3df4-9878-42e9-a3d6-d025d351016a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3c9d7d1dd1c450b836e39bc4face951'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:14:47.796614 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-163f3df4-9878-42e9-a3d6-d025d351016a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3c9d7d1dd1c450b836e39bc4face951'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:14:47.828590 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1b336f-1a64-435b-b0cf-b9acb49cc4b9 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:14:47.836793 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d831112c-2b54-4c28-b27a-d811bd4f9576 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '922ef7811aed450baff7f936011b0af1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:14:47.837417 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d831112c-2b54-4c28-b27a-d811bd4f9576 service placement] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:14:47.837697 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d831112c-2b54-4c28-b27a-d811bd4f9576 service placement] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:14:47.838010 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d831112c-2b54-4c28-b27a-d811bd4f9576 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:14:47.841121 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-d831112c-2b54-4c28-b27a-d811bd4f9576 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:14:47.857103 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d831112c-2b54-4c28-b27a-d811bd4f9576 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:14:47.860585 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d831112c-2b54-4c28-b27a-d811bd4f9576 service placement] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:14:47.882970 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1053/2140] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:14:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 03:14:47.885187 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d831112c-2b54-4c28-b27a-d811bd4f9576 service placement] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:14:47.888806 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-163f3df4-9878-42e9-a3d6-d025d351016a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3c9d7d1dd1c450b836e39bc4face951'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:14:47.890480 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-163f3df4-9878-42e9-a3d6-d025d351016a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3c9d7d1dd1c450b836e39bc4face951'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:14:47.896141 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-d831112c-2b54-4c28-b27a-d811bd4f9576 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:14:47.899690 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-163f3df4-9878-42e9-a3d6-d025d351016a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3c9d7d1dd1c450b836e39bc4face951'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:14:47.939190 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d831112c-2b54-4c28-b27a-d811bd4f9576 service placement] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:14:47.960891 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1088/2141] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:14:47 2022] DELETE /identity/v3/users/c3c9d7d1dd1c450b836e39bc4face951 => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:14:48.032664 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c237c376-ed43-41f4-80ef-e95bc69b795e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:14:48.033948 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c237c376-ed43-41f4-80ef-e95bc69b795e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:14:48.034858 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c237c376-ed43-41f4-80ef-e95bc69b795e admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:14:48.035276 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c237c376-ed43-41f4-80ef-e95bc69b795e admin admin] PATH_INFO: `/v3/users/410b7c9e93c94ce78d1b5c0e646307a2` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:14:48.037235 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c237c376-ed43-41f4-80ef-e95bc69b795e admin admin] RBAC: Authorizing `identity:delete_user(user_id=410b7c9e93c94ce78d1b5c0e646307a2)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:14:48.041456 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c237c376-ed43-41f4-80ef-e95bc69b795e admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:14:48.265265 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-c237c376-ed43-41f4-80ef-e95bc69b795e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '410b7c9e93c94ce78d1b5c0e646307a2'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:14:48.275286 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:14:48.283174 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-c237c376-ed43-41f4-80ef-e95bc69b795e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '410b7c9e93c94ce78d1b5c0e646307a2'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:14:48.306839 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-c237c376-ed43-41f4-80ef-e95bc69b795e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '410b7c9e93c94ce78d1b5c0e646307a2'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:14:48.308259 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-c237c376-ed43-41f4-80ef-e95bc69b795e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '410b7c9e93c94ce78d1b5c0e646307a2'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:14:48.389154 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-c237c376-ed43-41f4-80ef-e95bc69b795e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '410b7c9e93c94ce78d1b5c0e646307a2'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:14:48.403280 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1054/2142] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:14:47 2022] DELETE /identity/v3/users/410b7c9e93c94ce78d1b5c0e646307a2 => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:14:48.554296 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163f3df4-9878-42e9-a3d6-d025d351016a admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:14:48.556975 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-163f3df4-9878-42e9-a3d6-d025d351016a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:14:48.579089 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163f3df4-9878-42e9-a3d6-d025d351016a admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:14:48.665015 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fa5dc4-91dc-40c4-abfb-b455ca94c8cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:14:48.666011 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa5dc4-91dc-40c4-abfb-b455ca94c8cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:14:48.667552 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa5dc4-91dc-40c4-abfb-b455ca94c8cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:14:48.668011 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa5dc4-91dc-40c4-abfb-b455ca94c8cc admin admin] PATH_INFO: `/v3/projects/a4534457a4d64a79b0cd39461409d95a` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:14:48.670955 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fa5dc4-91dc-40c4-abfb-b455ca94c8cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4534457a4d64a79b0cd39461409d95a)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:14:48.673895 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fa5dc4-91dc-40c4-abfb-b455ca94c8cc admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:14:48.746395 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c237c376-ed43-41f4-80ef-e95bc69b795e admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:14:48.747352 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-c237c376-ed43-41f4-80ef-e95bc69b795e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:14:48.770164 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c237c376-ed43-41f4-80ef-e95bc69b795e admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:14:48.775223 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e1d151-2bb7-4c85-936c-d7f4a680917e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:14:48.775917 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e1d151-2bb7-4c85-936c-d7f4a680917e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:14:48.776202 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e1d151-2bb7-4c85-936c-d7f4a680917e admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:14:48.776524 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e1d151-2bb7-4c85-936c-d7f4a680917e admin admin] PATH_INFO: `/v3/projects/20fb24be77624e59b8560c7863ca4a46` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:14:48.777670 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e1d151-2bb7-4c85-936c-d7f4a680917e admin admin] RBAC: Authorizing `identity:delete_project(project_id=20fb24be77624e59b8560c7863ca4a46)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:14:48.779930 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e1d151-2bb7-4c85-936c-d7f4a680917e admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:14:48.827144 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-19fa5dc4-91dc-40c4-abfb-b455ca94c8cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4534457a4d64a79b0cd39461409d95a', 'request_id': 'req-19fa5dc4-91dc-40c4-abfb-b455ca94c8cc'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:14:48.840715 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-19fa5dc4-91dc-40c4-abfb-b455ca94c8cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4534457a4d64a79b0cd39461409d95a', 'request_id': 'req-19fa5dc4-91dc-40c4-abfb-b455ca94c8cc'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:14:48.921427 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-19fa5dc4-91dc-40c4-abfb-b455ca94c8cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4534457a4d64a79b0cd39461409d95a', 'request_id': 'req-19fa5dc4-91dc-40c4-abfb-b455ca94c8cc'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:14:48.939858 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-19fa5dc4-91dc-40c4-abfb-b455ca94c8cc admin admin] The token cache is being invalidate because project a4534457a4d64a79b0cd39461409d95a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81517) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:14:48.940177 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-19fa5dc4-91dc-40c4-abfb-b455ca94c8cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:14:48.940964 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-19fa5dc4-91dc-40c4-abfb-b455ca94c8cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:14:48.947526 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1089/2143] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:14:48 2022] DELETE /identity/v3/projects/a4534457a4d64a79b0cd39461409d95a => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:14:48.949843 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ebc4f5-366a-4f4f-9521-4c9656e07ad0 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:14:48.950145 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ebc4f5-366a-4f4f-9521-4c9656e07ad0 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:14:48.950754 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ebc4f5-366a-4f4f-9521-4c9656e07ad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:14:48.955232 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-07e1d151-2bb7-4c85-936c-d7f4a680917e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20fb24be77624e59b8560c7863ca4a46', 'request_id': 'req-07e1d151-2bb7-4c85-936c-d7f4a680917e'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:14:48.963336 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-07e1d151-2bb7-4c85-936c-d7f4a680917e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20fb24be77624e59b8560c7863ca4a46', 'request_id': 'req-07e1d151-2bb7-4c85-936c-d7f4a680917e'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:14:48.973138 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:14:49.007617 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-b7ebc4f5-366a-4f4f-9521-4c9656e07ad0 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:14:49.008919 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-b7ebc4f5-366a-4f4f-9521-4c9656e07ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:14:49.064915 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-07e1d151-2bb7-4c85-936c-d7f4a680917e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20fb24be77624e59b8560c7863ca4a46', 'request_id': 'req-07e1d151-2bb7-4c85-936c-d7f4a680917e'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:14:49.078218 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-07e1d151-2bb7-4c85-936c-d7f4a680917e admin admin] The token cache is being invalidate because project 20fb24be77624e59b8560c7863ca4a46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:14:49.078564 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-07e1d151-2bb7-4c85-936c-d7f4a680917e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:14:49.079225 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-07e1d151-2bb7-4c85-936c-d7f4a680917e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:14:49.081498 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1055/2144] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:14:48 2022] DELETE /identity/v3/projects/20fb24be77624e59b8560c7863ca4a46 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:14:49.084662 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1090/2145] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:14:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:14:49.087642 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b805e40e-1895-400c-acee-92dd6c285875 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:14:49.087934 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b805e40e-1895-400c-acee-92dd6c285875 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:14:49.088206 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b805e40e-1895-400c-acee-92dd6c285875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:14:49.092351 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c36b493-3fc2-43af-bed7-9fcb7e688e1f None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:14:49.093212 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c36b493-3fc2-43af-bed7-9fcb7e688e1f None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:14:49.093674 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c36b493-3fc2-43af-bed7-9fcb7e688e1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:14:49.118478 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-b805e40e-1895-400c-acee-92dd6c285875 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:14:49.120138 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-7c36b493-3fc2-43af-bed7-9fcb7e688e1f None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:14:49.121517 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-7c36b493-3fc2-43af-bed7-9fcb7e688e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:14:49.121663 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-b805e40e-1895-400c-acee-92dd6c285875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:14:49.134704 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1056/2146] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:14:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:14:49.135589 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1091/2147] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:14:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:14:49.139525 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b805e40e-1895-400c-acee-92dd6c285875 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:14:49.141941 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b84eaf-1e05-441b-ac89-d24019683c7f None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:14:49.142450 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b84eaf-1e05-441b-ac89-d24019683c7f None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:14:49.142999 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b84eaf-1e05-441b-ac89-d24019683c7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:14:49.158346 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b805e40e-1895-400c-acee-92dd6c285875 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:14:49.164517 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0ac90c-7dad-42d2-ace5-7f1da2e8c6c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:14:49.165365 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0ac90c-7dad-42d2-ace5-7f1da2e8c6c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:14:49.166358 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0ac90c-7dad-42d2-ace5-7f1da2e8c6c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:14:49.166827 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0ac90c-7dad-42d2-ace5-7f1da2e8c6c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:14:49.167666 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0ac90c-7dad-42d2-ace5-7f1da2e8c6c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 03:14:49.168395 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-70b84eaf-1e05-441b-ac89-d24019683c7f None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:14:49.168841 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0ac90c-7dad-42d2-ace5-7f1da2e8c6c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:14:49.170784 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-70b84eaf-1e05-441b-ac89-d24019683c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:14:49.171410 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0ac90c-7dad-42d2-ace5-7f1da2e8c6c1 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:14:49.187016 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1092/2148] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:14:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 03:14:49.189954 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1057/2149] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Oct 14 03:14:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 03:14:49.191088 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b84eaf-1e05-441b-ac89-d24019683c7f None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:14:49.192261 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0ac90c-7dad-42d2-ace5-7f1da2e8c6c1 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:14:49.201433 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b84eaf-1e05-441b-ac89-d24019683c7f None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:14:49.202356 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0ac90c-7dad-42d2-ace5-7f1da2e8c6c1 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:14:49.206085 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cebfce-bc63-45f9-b1e0-0be3f51bd2f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:14:49.206692 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cebfce-bc63-45f9-b1e0-0be3f51bd2f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:14:49.206971 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cebfce-bc63-45f9-b1e0-0be3f51bd2f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:14:49.207082 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035cdffb-fd6d-404e-8f4f-bbd03d1917c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:14:49.207289 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cebfce-bc63-45f9-b1e0-0be3f51bd2f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:14:49.207663 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035cdffb-fd6d-404e-8f4f-bbd03d1917c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:14:49.207937 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035cdffb-fd6d-404e-8f4f-bbd03d1917c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:14:49.208073 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cebfce-bc63-45f9-b1e0-0be3f51bd2f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 03:14:49.208232 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035cdffb-fd6d-404e-8f4f-bbd03d1917c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:14:49.208344 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cebfce-bc63-45f9-b1e0-0be3f51bd2f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:14:49.208835 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035cdffb-fd6d-404e-8f4f-bbd03d1917c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:14:49.210566 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cebfce-bc63-45f9-b1e0-0be3f51bd2f2 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:14:49.210772 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035cdffb-fd6d-404e-8f4f-bbd03d1917c4 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:14:49.230355 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1093/2150] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Oct 14 03:14:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 03:14:49.232052 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cebfce-bc63-45f9-b1e0-0be3f51bd2f2 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:14:49.252471 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cebfce-bc63-45f9-b1e0-0be3f51bd2f2 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:14:49.259398 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198360e8-a4eb-4cde-a77c-170abb6607c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:14:49.260173 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198360e8-a4eb-4cde-a77c-170abb6607c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:14:49.260679 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198360e8-a4eb-4cde-a77c-170abb6607c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:14:49.261024 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198360e8-a4eb-4cde-a77c-170abb6607c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:14:49.261818 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198360e8-a4eb-4cde-a77c-170abb6607c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:14:49.263883 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198360e8-a4eb-4cde-a77c-170abb6607c3 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:14:49.335277 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1058/2151] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:14:49 2022] POST /identity/v3/projects => generated 375 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 03:14:49.339358 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035cdffb-fd6d-404e-8f4f-bbd03d1917c4 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:14:49.368040 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1094/2152] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:14:49 2022] POST /identity/v3/projects => generated 379 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 03:14:49.378083 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198360e8-a4eb-4cde-a77c-170abb6607c3 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:14:49.407376 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035cdffb-fd6d-404e-8f4f-bbd03d1917c4 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:14:49.412750 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:14:49.446818 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198360e8-a4eb-4cde-a77c-170abb6607c3 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:14:49.453029 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e378c3-7c03-46b0-b8c8-70130a43e390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:14:49.453859 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e378c3-7c03-46b0-b8c8-70130a43e390 admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:14:49.454111 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e378c3-7c03-46b0-b8c8-70130a43e390 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:14:49.454422 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e378c3-7c03-46b0-b8c8-70130a43e390 admin admin] PATH_INFO: `/v3/users` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:14:49.457897 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e378c3-7c03-46b0-b8c8-70130a43e390 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:14:49.457897 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e378c3-7c03-46b0-b8c8-70130a43e390 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:14:49.471568 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3e7791-1b76-4a1b-9987-7f23fc2e7c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:14:49.472272 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3e7791-1b76-4a1b-9987-7f23fc2e7c0e admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:14:49.472553 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3e7791-1b76-4a1b-9987-7f23fc2e7c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:14:49.472825 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3e7791-1b76-4a1b-9987-7f23fc2e7c0e admin admin] PATH_INFO: `/v3/users` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:14:49.473589 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3e7791-1b76-4a1b-9987-7f23fc2e7c0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:14:49.475595 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3e7791-1b76-4a1b-9987-7f23fc2e7c0e admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:14:49.560532 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1095/2153] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:14:49 2022] POST /identity/v3/users => generated 290 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:14:49.561732 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e378c3-7c03-46b0-b8c8-70130a43e390 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:14:49.571647 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e378c3-7c03-46b0-b8c8-70130a43e390 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:14:49.577245 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55675cba-2683-4666-b9b8-46b65e1c0eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:14:49.577803 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55675cba-2683-4666-b9b8-46b65e1c0eab admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:14:49.578087 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55675cba-2683-4666-b9b8-46b65e1c0eab admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:14:49.578589 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55675cba-2683-4666-b9b8-46b65e1c0eab admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:14:49.579046 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55675cba-2683-4666-b9b8-46b65e1c0eab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:14:49.580658 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55675cba-2683-4666-b9b8-46b65e1c0eab admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:14:49.588726 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1059/2154] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:14:49 2022] POST /identity/v3/users => generated 288 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 03:14:49.593803 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1096/2155] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:14:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 03:14:49.593949 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3e7791-1b76-4a1b-9987-7f23fc2e7c0e admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:14:49.595823 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55675cba-2683-4666-b9b8-46b65e1c0eab admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:14:49.603331 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3e7791-1b76-4a1b-9987-7f23fc2e7c0e admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:14:49.609626 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55675cba-2683-4666-b9b8-46b65e1c0eab admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:14:49.609809 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fd5190-feb2-4417-8513-e11703d01d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:14:49.609809 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fd5190-feb2-4417-8513-e11703d01d01 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:14:49.609809 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fd5190-feb2-4417-8513-e11703d01d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:14:49.609809 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fd5190-feb2-4417-8513-e11703d01d01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:14:49.610186 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fd5190-feb2-4417-8513-e11703d01d01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:14:49.611379 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fd5190-feb2-4417-8513-e11703d01d01 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:14:49.612099 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21823ae1-4687-497b-9bea-58ca4ecf18df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:14:49.612692 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21823ae1-4687-497b-9bea-58ca4ecf18df admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:14:49.612980 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21823ae1-4687-497b-9bea-58ca4ecf18df admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:14:49.613255 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21823ae1-4687-497b-9bea-58ca4ecf18df admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:14:49.613978 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21823ae1-4687-497b-9bea-58ca4ecf18df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:14:49.615567 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21823ae1-4687-497b-9bea-58ca4ecf18df admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:14:49.622284 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1060/2156] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:14:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 03:14:49.626328 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fd5190-feb2-4417-8513-e11703d01d01 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:14:49.626473 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1097/2157] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:14:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 03:14:49.630047 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21823ae1-4687-497b-9bea-58ca4ecf18df admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:14:49.635820 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fd5190-feb2-4417-8513-e11703d01d01 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:14:49.640489 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21823ae1-4687-497b-9bea-58ca4ecf18df admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:14:49.643615 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84ff6d6-ae97-4f7e-b6d8-33eadad96b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:14:49.645523 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84ff6d6-ae97-4f7e-b6d8-33eadad96b3b admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:14:49.645523 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84ff6d6-ae97-4f7e-b6d8-33eadad96b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:14:49.645523 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84ff6d6-ae97-4f7e-b6d8-33eadad96b3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:14:49.646080 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84ff6d6-ae97-4f7e-b6d8-33eadad96b3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:14:49.646764 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdcee6f-e2d7-4b4a-84e6-be2a861f0888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:14:49.647382 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84ff6d6-ae97-4f7e-b6d8-33eadad96b3b admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:14:49.647782 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdcee6f-e2d7-4b4a-84e6-be2a861f0888 admin admin] REQUEST_METHOD: `PUT` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:14:49.648067 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdcee6f-e2d7-4b4a-84e6-be2a861f0888 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:14:49.648346 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdcee6f-e2d7-4b4a-84e6-be2a861f0888 admin admin] PATH_INFO: `/v3/projects/f454dcb9b35b4ae797d048424e1065a1/users/7ceb22bf95c243b98bb517abe40ce583/roles/9bd0470b544240209a420f640c79c06c` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:14:49.651730 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:14:49.661834 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1061/2158] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:14:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 03:14:49.663555 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84ff6d6-ae97-4f7e-b6d8-33eadad96b3b admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:14:49.666384 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdcee6f-e2d7-4b4a-84e6-be2a861f0888 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f454dcb9b35b4ae797d048424e1065a1, user_id=7ceb22bf95c243b98bb517abe40ce583, role_id=9bd0470b544240209a420f640c79c06c)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:14:49.669077 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdcee6f-e2d7-4b4a-84e6-be2a861f0888 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:14:49.676177 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84ff6d6-ae97-4f7e-b6d8-33eadad96b3b admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:14:49.680992 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52944e06-c54c-4f1e-ad04-a901d9ca0422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:14:49.681643 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52944e06-c54c-4f1e-ad04-a901d9ca0422 admin admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:14:49.681953 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52944e06-c54c-4f1e-ad04-a901d9ca0422 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:14:49.682223 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52944e06-c54c-4f1e-ad04-a901d9ca0422 admin admin] PATH_INFO: `/v3/projects/1270f42959814dd0af340a1df0dee17c/users/00d23c554e5e464380b8f9ceb4cc0f2f/roles/9bd0470b544240209a420f640c79c06c` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:14:49.699523 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52944e06-c54c-4f1e-ad04-a901d9ca0422 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1270f42959814dd0af340a1df0dee17c, user_id=00d23c554e5e464380b8f9ceb4cc0f2f, role_id=9bd0470b544240209a420f640c79c06c)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:14:49.702931 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52944e06-c54c-4f1e-ad04-a901d9ca0422 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:14:49.785616 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1098/2159] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:14:49 2022] PUT /identity/v3/projects/f454dcb9b35b4ae797d048424e1065a1/users/7ceb22bf95c243b98bb517abe40ce583/roles/9bd0470b544240209a420f640c79c06c => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 03:14:49.797394 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96465aa-65a4-4704-9b77-a9d75232bb8f None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:14:49.798815 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96465aa-65a4-4704-9b77-a9d75232bb8f None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:14:49.799557 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96465aa-65a4-4704-9b77-a9d75232bb8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:14:49.830298 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1062/2160] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:14:49 2022] PUT /identity/v3/projects/1270f42959814dd0af340a1df0dee17c/users/00d23c554e5e464380b8f9ceb4cc0f2f/roles/9bd0470b544240209a420f640c79c06c => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 03:14:49.857379 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a515b2-48d1-4846-871f-588ec92bff0a None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:14:49.858313 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a515b2-48d1-4846-871f-588ec92bff0a None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:14:49.858760 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a515b2-48d1-4846-871f-588ec92bff0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:14:49.867717 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-d96465aa-65a4-4704-9b77-a9d75232bb8f None None] MFA Rules not processed for user `7ceb22bf95c243b98bb517abe40ce583`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:14:49.870956 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-d96465aa-65a4-4704-9b77-a9d75232bb8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:14:49.922683 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-f6a515b2-48d1-4846-871f-588ec92bff0a None None] MFA Rules not processed for user `00d23c554e5e464380b8f9ceb4cc0f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:14:49.928851 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-f6a515b2-48d1-4846-871f-588ec92bff0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:14:50.001457 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1099/2161] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:14:49 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 03:14:50.002651 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96465aa-65a4-4704-9b77-a9d75232bb8f None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:14:50.061970 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1063/2162] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:14:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:14:50.074041 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a515b2-48d1-4846-871f-588ec92bff0a None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:14:50.075592 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-f6a515b2-48d1-4846-871f-588ec92bff0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:14:50.123695 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96465aa-65a4-4704-9b77-a9d75232bb8f None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:14:50.129085 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cebf172-4b57-4908-ae19-ea8cf77ce151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:14:50.129847 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cebf172-4b57-4908-ae19-ea8cf77ce151 admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:14:50.130136 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cebf172-4b57-4908-ae19-ea8cf77ce151 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:14:50.130756 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cebf172-4b57-4908-ae19-ea8cf77ce151 admin admin] PATH_INFO: `/v3/projects` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:14:50.131040 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cebf172-4b57-4908-ae19-ea8cf77ce151 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:14:50.133145 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cebf172-4b57-4908-ae19-ea8cf77ce151 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:14:50.138402 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:14:50.174069 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a515b2-48d1-4846-871f-588ec92bff0a None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:14:50.177856 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4313485b-244d-43d6-a55f-164b827f0074 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab599d98b0624a2e89090312bcab713f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:14:50.178463 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4313485b-244d-43d6-a55f-164b827f0074 service cinder] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:14:50.178747 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4313485b-244d-43d6-a55f-164b827f0074 service cinder] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:14:50.179023 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4313485b-244d-43d6-a55f-164b827f0074 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:14:50.187852 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4313485b-244d-43d6-a55f-164b827f0074 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:14:50.189284 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4313485b-244d-43d6-a55f-164b827f0074 service cinder] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:14:50.198068 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1064/2163] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:14:50 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 03:14:50.298516 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1100/2164] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:14:50 2022] POST /identity/v3/projects => generated 381 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:14:50.303138 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4313485b-244d-43d6-a55f-164b827f0074 service cinder] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:14:50.319044 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4313485b-244d-43d6-a55f-164b827f0074 service cinder] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:14:50.388807 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d00064d-ac27-4597-b6f1-195ca87a359b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:14:50.389422 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d00064d-ac27-4597-b6f1-195ca87a359b admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:14:50.389808 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d00064d-ac27-4597-b6f1-195ca87a359b admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:14:50.390016 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d00064d-ac27-4597-b6f1-195ca87a359b admin admin] PATH_INFO: `/v3/users` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:14:50.390700 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d00064d-ac27-4597-b6f1-195ca87a359b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:14:50.392640 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d00064d-ac27-4597-b6f1-195ca87a359b admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:14:50.426275 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cebf172-4b57-4908-ae19-ea8cf77ce151 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:14:50.427945 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-5cebf172-4b57-4908-ae19-ea8cf77ce151 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:14:50.496654 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1065/2165] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:14:50 2022] POST /identity/v3/users => generated 297 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:14:50.499852 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:14:50.501070 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d00064d-ac27-4597-b6f1-195ca87a359b admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:14:50.506312 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cebf172-4b57-4908-ae19-ea8cf77ce151 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:14:50.510620 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d00064d-ac27-4597-b6f1-195ca87a359b admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:14:50.511890 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45396a23-d9dd-4d84-9f12-a9164c15ae30 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '922ef7811aed450baff7f936011b0af1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:14:50.512495 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45396a23-d9dd-4d84-9f12-a9164c15ae30 service placement] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:14:50.512777 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45396a23-d9dd-4d84-9f12-a9164c15ae30 service placement] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:14:50.513051 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45396a23-d9dd-4d84-9f12-a9164c15ae30 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:14:50.514824 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-45396a23-d9dd-4d84-9f12-a9164c15ae30 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:14:50.515681 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca799fc-5807-4716-b90e-fd19bef54494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:14:50.516336 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca799fc-5807-4716-b90e-fd19bef54494 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:14:50.516690 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca799fc-5807-4716-b90e-fd19bef54494 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:14:50.517030 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca799fc-5807-4716-b90e-fd19bef54494 admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:14:50.517746 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca799fc-5807-4716-b90e-fd19bef54494 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:14:50.519503 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca799fc-5807-4716-b90e-fd19bef54494 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:14:50.527190 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45396a23-d9dd-4d84-9f12-a9164c15ae30 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:14:50.528795 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45396a23-d9dd-4d84-9f12-a9164c15ae30 service placement] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:14:50.534144 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1066/2166] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:14:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 03:14:50.536789 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca799fc-5807-4716-b90e-fd19bef54494 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:14:50.539601 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1101/2167] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:14:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:14:50.547540 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca799fc-5807-4716-b90e-fd19bef54494 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:14:50.552707 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8178a275-9788-494a-ae42-137f72d87426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:14:50.553557 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8178a275-9788-494a-ae42-137f72d87426 admin admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:14:50.553945 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8178a275-9788-494a-ae42-137f72d87426 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:14:50.554350 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8178a275-9788-494a-ae42-137f72d87426 admin admin] PATH_INFO: `/v3/projects/a96a9826450349618792e6a179fec75d/users/2a522089e1c545eb928adb65e7d3b41f/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:14:50.570354 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45396a23-d9dd-4d84-9f12-a9164c15ae30 service placement] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:14:50.572932 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-45396a23-d9dd-4d84-9f12-a9164c15ae30 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:14:50.588858 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45396a23-d9dd-4d84-9f12-a9164c15ae30 service placement] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:14:50.589965 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8178a275-9788-494a-ae42-137f72d87426 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a96a9826450349618792e6a179fec75d, user_id=2a522089e1c545eb928adb65e7d3b41f, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:14:50.593290 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8178a275-9788-494a-ae42-137f72d87426 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:14:50.595118 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e077eb9e-9e7f-4240-aec0-13d0270a771d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:14:50.596066 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e077eb9e-9e7f-4240-aec0-13d0270a771d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:14:50.596621 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e077eb9e-9e7f-4240-aec0-13d0270a771d admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:14:50.596903 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e077eb9e-9e7f-4240-aec0-13d0270a771d admin admin] PATH_INFO: `/v3/users/fb1c279c2f7f404ab9c846a3ba594886` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:14:50.598226 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e077eb9e-9e7f-4240-aec0-13d0270a771d admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb1c279c2f7f404ab9c846a3ba594886)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:14:50.600098 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e077eb9e-9e7f-4240-aec0-13d0270a771d admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:14:50.721101 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1067/2168] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:14:50 2022] PUT /identity/v3/projects/a96a9826450349618792e6a179fec75d/users/2a522089e1c545eb928adb65e7d3b41f/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 03:14:50.725213 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8178a275-9788-494a-ae42-137f72d87426 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:14:50.786496 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8178a275-9788-494a-ae42-137f72d87426 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:14:50.791212 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe0625b-3286-4644-a52f-93219d576732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:14:50.791832 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe0625b-3286-4644-a52f-93219d576732 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:14:50.792112 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe0625b-3286-4644-a52f-93219d576732 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:14:50.792407 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe0625b-3286-4644-a52f-93219d576732 admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:14:50.793070 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe0625b-3286-4644-a52f-93219d576732 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:14:50.795406 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe0625b-3286-4644-a52f-93219d576732 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:14:50.805101 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1068/2169] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:14:50 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 03:14:50.809040 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe0625b-3286-4644-a52f-93219d576732 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:14:50.818455 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe0625b-3286-4644-a52f-93219d576732 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:14:50.823273 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ec02c7-0ac8-4a38-a0c0-26f32557cefc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:14:50.823900 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ec02c7-0ac8-4a38-a0c0-26f32557cefc admin admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:14:50.824185 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ec02c7-0ac8-4a38-a0c0-26f32557cefc admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:14:50.824465 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ec02c7-0ac8-4a38-a0c0-26f32557cefc admin admin] PATH_INFO: `/v3/projects/a96a9826450349618792e6a179fec75d/users/2a522089e1c545eb928adb65e7d3b41f/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:14:50.826457 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ec02c7-0ac8-4a38-a0c0-26f32557cefc admin admin] RBAC: Authorizing `identity:create_grant(project_id=a96a9826450349618792e6a179fec75d, user_id=2a522089e1c545eb928adb65e7d3b41f, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:14:50.829090 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ec02c7-0ac8-4a38-a0c0-26f32557cefc admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:14:50.829901 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-e077eb9e-9e7f-4240-aec0-13d0270a771d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb1c279c2f7f404ab9c846a3ba594886'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:14:50.830653 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-e077eb9e-9e7f-4240-aec0-13d0270a771d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb1c279c2f7f404ab9c846a3ba594886'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:14:50.839490 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1069/2170] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:14:50 2022] PUT /identity/v3/projects/a96a9826450349618792e6a179fec75d/users/2a522089e1c545eb928adb65e7d3b41f/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 03:14:50.850170 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74e507c-db56-4c33-b9c6-fe4c85bab386 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:14:50.850460 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74e507c-db56-4c33-b9c6-fe4c85bab386 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:14:50.850735 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74e507c-db56-4c33-b9c6-fe4c85bab386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:14:50.859041 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:14:50.879189 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-d74e507c-db56-4c33-b9c6-fe4c85bab386 None None] MFA Rules not processed for user `2a522089e1c545eb928adb65e7d3b41f`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:14:50.880413 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-d74e507c-db56-4c33-b9c6-fe4c85bab386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:14:50.891110 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-e077eb9e-9e7f-4240-aec0-13d0270a771d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb1c279c2f7f404ab9c846a3ba594886'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:14:50.891807 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-e077eb9e-9e7f-4240-aec0-13d0270a771d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb1c279c2f7f404ab9c846a3ba594886'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:14:50.899325 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-e077eb9e-9e7f-4240-aec0-13d0270a771d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb1c279c2f7f404ab9c846a3ba594886'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:14:50.917797 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1102/2171] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:14:50 2022] DELETE /identity/v3/users/fb1c279c2f7f404ab9c846a3ba594886 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 03:14:50.951085 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1070/2172] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:14:50 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 03:14:50.962888 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e077eb9e-9e7f-4240-aec0-13d0270a771d admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:14:50.964573 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-e077eb9e-9e7f-4240-aec0-13d0270a771d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:14:51.046530 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e077eb9e-9e7f-4240-aec0-13d0270a771d admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:14:51.050364 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62ea2dc-9fc6-4a76-b18d-416cf3dfd01c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab599d98b0624a2e89090312bcab713f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:14:51.051254 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62ea2dc-9fc6-4a76-b18d-416cf3dfd01c service cinder] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:14:51.051604 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62ea2dc-9fc6-4a76-b18d-416cf3dfd01c service cinder] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:14:51.051944 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62ea2dc-9fc6-4a76-b18d-416cf3dfd01c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:14:51.124324 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62ea2dc-9fc6-4a76-b18d-416cf3dfd01c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:14:51.125989 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62ea2dc-9fc6-4a76-b18d-416cf3dfd01c service cinder] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:14:51.135702 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1103/2173] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:14:50 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:14:51.189852 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74e507c-db56-4c33-b9c6-fe4c85bab386 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:14:51.191495 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-d74e507c-db56-4c33-b9c6-fe4c85bab386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:14:51.264987 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74e507c-db56-4c33-b9c6-fe4c85bab386 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:14:51.320230 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c77cad3-6710-4091-8c8a-d4ec61a4844e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:14:51.320876 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c77cad3-6710-4091-8c8a-d4ec61a4844e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:14:51.321162 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c77cad3-6710-4091-8c8a-d4ec61a4844e admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:14:51.321439 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c77cad3-6710-4091-8c8a-d4ec61a4844e admin admin] PATH_INFO: `/v3/projects/5470d8ee28f74b4eb385a90c33a514b8` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:14:51.322633 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c77cad3-6710-4091-8c8a-d4ec61a4844e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5470d8ee28f74b4eb385a90c33a514b8)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:14:51.322783 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62ea2dc-9fc6-4a76-b18d-416cf3dfd01c service cinder] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:14:51.324575 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c77cad3-6710-4091-8c8a-d4ec61a4844e admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:14:51.332062 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62ea2dc-9fc6-4a76-b18d-416cf3dfd01c service cinder] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:14:51.335750 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f51d0-15f3-4274-b1ba-2abf39359414 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab599d98b0624a2e89090312bcab713f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:14:51.336345 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f51d0-15f3-4274-b1ba-2abf39359414 service cinder] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:14:51.336628 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f51d0-15f3-4274-b1ba-2abf39359414 service cinder] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:14:51.336959 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f51d0-15f3-4274-b1ba-2abf39359414 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:14:51.338752 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-528f51d0-15f3-4274-b1ba-2abf39359414 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:14:51.348899 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:14:51.398058 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528f51d0-15f3-4274-b1ba-2abf39359414 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:14:51.399488 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528f51d0-15f3-4274-b1ba-2abf39359414 service cinder] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:14:51.408211 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1104/2174] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:14:51 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 03:14:51.496623 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-5c77cad3-6710-4091-8c8a-d4ec61a4844e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5470d8ee28f74b4eb385a90c33a514b8', 'request_id': 'req-5c77cad3-6710-4091-8c8a-d4ec61a4844e'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:14:51.503272 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-5c77cad3-6710-4091-8c8a-d4ec61a4844e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5470d8ee28f74b4eb385a90c33a514b8', 'request_id': 'req-5c77cad3-6710-4091-8c8a-d4ec61a4844e'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:14:51.596622 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-5c77cad3-6710-4091-8c8a-d4ec61a4844e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5470d8ee28f74b4eb385a90c33a514b8', 'request_id': 'req-5c77cad3-6710-4091-8c8a-d4ec61a4844e'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:14:51.607156 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-5c77cad3-6710-4091-8c8a-d4ec61a4844e admin admin] The token cache is being invalidate because project 5470d8ee28f74b4eb385a90c33a514b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:14:51.607445 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-5c77cad3-6710-4091-8c8a-d4ec61a4844e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:14:51.608070 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-5c77cad3-6710-4091-8c8a-d4ec61a4844e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:14:51.609561 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1071/2175] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:14:51 2022] DELETE /identity/v3/projects/5470d8ee28f74b4eb385a90c33a514b8 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:14:51.614837 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edac441c-4991-4dea-a3b5-38fea6168e0e None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:14:51.615306 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edac441c-4991-4dea-a3b5-38fea6168e0e None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:14:51.615417 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edac441c-4991-4dea-a3b5-38fea6168e0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:14:51.635391 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-edac441c-4991-4dea-a3b5-38fea6168e0e None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:14:51.636594 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-edac441c-4991-4dea-a3b5-38fea6168e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:14:51.683114 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:14:51.695673 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1105/2176] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:14:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:14:51.705231 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f2bd24-dbb9-4dfe-9ceb-16ed44f04f65 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:14:51.705516 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f2bd24-dbb9-4dfe-9ceb-16ed44f04f65 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:14:51.705822 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f2bd24-dbb9-4dfe-9ceb-16ed44f04f65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:14:51.725924 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-30f2bd24-dbb9-4dfe-9ceb-16ed44f04f65 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:14:51.727157 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-30f2bd24-dbb9-4dfe-9ceb-16ed44f04f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:14:51.735984 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1072/2177] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:14:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:14:51.739417 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edac441c-4991-4dea-a3b5-38fea6168e0e None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:14:51.749977 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edac441c-4991-4dea-a3b5-38fea6168e0e None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:14:51.754474 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfc7d0a-215b-4c88-bb12-fd516a50e54d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:14:51.755063 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc7d0a-215b-4c88-bb12-fd516a50e54d admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:14:51.755429 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc7d0a-215b-4c88-bb12-fd516a50e54d admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:14:51.755706 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc7d0a-215b-4c88-bb12-fd516a50e54d admin admin] PATH_INFO: `/v3/domains` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:14:51.756349 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfc7d0a-215b-4c88-bb12-fd516a50e54d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 03:14:51.756663 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfc7d0a-215b-4c88-bb12-fd516a50e54d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:14:51.758287 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfc7d0a-215b-4c88-bb12-fd516a50e54d admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:14:51.771068 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1106/2178] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Oct 14 03:14:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 03:14:51.775498 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f2bd24-dbb9-4dfe-9ceb-16ed44f04f65 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:14:51.784979 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f2bd24-dbb9-4dfe-9ceb-16ed44f04f65 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:14:51.789639 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975282f0-c004-4af8-b46d-6329dd56c918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:14:51.790300 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975282f0-c004-4af8-b46d-6329dd56c918 admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:14:51.790578 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975282f0-c004-4af8-b46d-6329dd56c918 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:14:51.790850 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975282f0-c004-4af8-b46d-6329dd56c918 admin admin] PATH_INFO: `/v3/projects` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:14:51.791480 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975282f0-c004-4af8-b46d-6329dd56c918 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:14:51.793415 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975282f0-c004-4af8-b46d-6329dd56c918 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:14:51.939769 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1073/2179] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:14:51 2022] POST /identity/v3/projects => generated 377 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:14:51.940855 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfc7d0a-215b-4c88-bb12-fd516a50e54d admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:14:51.968512 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfc7d0a-215b-4c88-bb12-fd516a50e54d admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:14:52.020384 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d7abfb-9d0e-4e0d-abe0-770129f49bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:14:52.021084 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d7abfb-9d0e-4e0d-abe0-770129f49bab admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:14:52.021367 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d7abfb-9d0e-4e0d-abe0-770129f49bab admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:14:52.021645 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d7abfb-9d0e-4e0d-abe0-770129f49bab admin admin] PATH_INFO: `/v3/users` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:14:52.022324 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d7abfb-9d0e-4e0d-abe0-770129f49bab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:14:52.024284 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d7abfb-9d0e-4e0d-abe0-770129f49bab admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:14:52.105746 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1107/2180] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:14:51 2022] POST /identity/v3/users => generated 289 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:14:52.107019 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975282f0-c004-4af8-b46d-6329dd56c918 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:14:52.118275 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975282f0-c004-4af8-b46d-6329dd56c918 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:14:52.124405 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2256eea0-84d3-4c52-8dde-c01bd650d860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:14:52.125028 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2256eea0-84d3-4c52-8dde-c01bd650d860 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:14:52.125312 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2256eea0-84d3-4c52-8dde-c01bd650d860 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:14:52.128479 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2256eea0-84d3-4c52-8dde-c01bd650d860 admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:14:52.128479 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2256eea0-84d3-4c52-8dde-c01bd650d860 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:14:52.128479 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2256eea0-84d3-4c52-8dde-c01bd650d860 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:14:52.138155 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1074/2181] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:14:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:14:52.142349 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d7abfb-9d0e-4e0d-abe0-770129f49bab admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:14:52.152703 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d7abfb-9d0e-4e0d-abe0-770129f49bab admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:14:52.158047 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d3268d-3030-4bea-a816-588facd4dd91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:14:52.158642 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d3268d-3030-4bea-a816-588facd4dd91 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:14:52.158929 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d3268d-3030-4bea-a816-588facd4dd91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:14:52.159208 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d3268d-3030-4bea-a816-588facd4dd91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:14:52.159852 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d3268d-3030-4bea-a816-588facd4dd91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:14:52.161456 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d3268d-3030-4bea-a816-588facd4dd91 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:14:52.172576 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1108/2182] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:14:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:14:52.176913 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2256eea0-84d3-4c52-8dde-c01bd650d860 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:14:52.186930 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2256eea0-84d3-4c52-8dde-c01bd650d860 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:14:52.191547 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccc925d-3da5-485f-84d6-4fdae394bda9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:14:52.192270 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccc925d-3da5-485f-84d6-4fdae394bda9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:14:52.192553 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccc925d-3da5-485f-84d6-4fdae394bda9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:14:52.192828 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccc925d-3da5-485f-84d6-4fdae394bda9 admin admin] PATH_INFO: `/v3/projects/f5763416b30a4109a4859d989261e6ad/users/63a3c5f18b7b4cb0b411c68c1ef99afa/roles/9bd0470b544240209a420f640c79c06c` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:14:52.207322 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccc925d-3da5-485f-84d6-4fdae394bda9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5763416b30a4109a4859d989261e6ad, user_id=63a3c5f18b7b4cb0b411c68c1ef99afa, role_id=9bd0470b544240209a420f640c79c06c)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:14:52.210140 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccc925d-3da5-485f-84d6-4fdae394bda9 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:14:52.327956 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1075/2183] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:14:52 2022] PUT /identity/v3/projects/f5763416b30a4109a4859d989261e6ad/users/63a3c5f18b7b4cb0b411c68c1ef99afa/roles/9bd0470b544240209a420f640c79c06c => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:14:52.338226 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a018e14-a4be-4641-891f-c05968dd73ee None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:14:52.338516 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a018e14-a4be-4641-891f-c05968dd73ee None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:14:52.338796 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a018e14-a4be-4641-891f-c05968dd73ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:14:52.367872 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-0a018e14-a4be-4641-891f-c05968dd73ee None None] MFA Rules not processed for user `63a3c5f18b7b4cb0b411c68c1ef99afa`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:14:52.369087 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-0a018e14-a4be-4641-891f-c05968dd73ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:14:52.373081 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:14:52.431742 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1109/2184] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:14:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:14:52.436427 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccc925d-3da5-485f-84d6-4fdae394bda9 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:14:52.501679 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccc925d-3da5-485f-84d6-4fdae394bda9 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:14:52.506499 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164b76d3-e3c1-4dbf-9c23-4bf3043969b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:14:52.507082 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164b76d3-e3c1-4dbf-9c23-4bf3043969b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:14:52.507366 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164b76d3-e3c1-4dbf-9c23-4bf3043969b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:14:52.507646 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164b76d3-e3c1-4dbf-9c23-4bf3043969b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:14:52.508305 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164b76d3-e3c1-4dbf-9c23-4bf3043969b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:14:52.510240 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164b76d3-e3c1-4dbf-9c23-4bf3043969b8 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:14:52.612773 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1076/2185] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:14:52 2022] POST /identity/v3/projects => generated 377 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:14:52.616799 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a018e14-a4be-4641-891f-c05968dd73ee None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:14:52.628523 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a018e14-a4be-4641-891f-c05968dd73ee None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:14:52.685096 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b336ec89-c23b-4c25-b1ed-f5bb27be39a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:14:52.685694 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b336ec89-c23b-4c25-b1ed-f5bb27be39a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:14:52.686003 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b336ec89-c23b-4c25-b1ed-f5bb27be39a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:14:52.686275 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b336ec89-c23b-4c25-b1ed-f5bb27be39a6 admin admin] PATH_INFO: `/v3/users` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:14:52.686992 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b336ec89-c23b-4c25-b1ed-f5bb27be39a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:14:52.688909 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b336ec89-c23b-4c25-b1ed-f5bb27be39a6 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:14:52.790713 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1110/2186] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:14:52 2022] POST /identity/v3/users => generated 295 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:14:52.794787 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164b76d3-e3c1-4dbf-9c23-4bf3043969b8 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:14:52.804919 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164b76d3-e3c1-4dbf-9c23-4bf3043969b8 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:14:52.809501 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f43dca-4fd2-4774-b49c-4ae198d15535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:14:52.810126 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f43dca-4fd2-4774-b49c-4ae198d15535 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:14:52.816571 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f43dca-4fd2-4774-b49c-4ae198d15535 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:14:52.816571 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f43dca-4fd2-4774-b49c-4ae198d15535 admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:14:52.816571 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f43dca-4fd2-4774-b49c-4ae198d15535 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:14:52.816571 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f43dca-4fd2-4774-b49c-4ae198d15535 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:14:52.823284 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1077/2187] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:14:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:14:52.827814 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b336ec89-c23b-4c25-b1ed-f5bb27be39a6 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:14:52.837808 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b336ec89-c23b-4c25-b1ed-f5bb27be39a6 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:14:52.843178 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6011182a-1b04-41ec-9bdf-10d99bb36519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:14:52.843809 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6011182a-1b04-41ec-9bdf-10d99bb36519 admin admin] REQUEST_METHOD: `PUT` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:14:52.844089 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6011182a-1b04-41ec-9bdf-10d99bb36519 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:14:52.844361 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6011182a-1b04-41ec-9bdf-10d99bb36519 admin admin] PATH_INFO: `/v3/projects/f04b00e4841d4f68a5dede354be864ff/users/4b255ae026fb4c519ceb54fa393a5718/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:14:52.859347 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6011182a-1b04-41ec-9bdf-10d99bb36519 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f04b00e4841d4f68a5dede354be864ff, user_id=4b255ae026fb4c519ceb54fa393a5718, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:14:52.862405 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6011182a-1b04-41ec-9bdf-10d99bb36519 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:14:52.957394 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1111/2188] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:14:52 2022] PUT /identity/v3/projects/f04b00e4841d4f68a5dede354be864ff/users/4b255ae026fb4c519ceb54fa393a5718/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:14:52.961447 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f43dca-4fd2-4774-b49c-4ae198d15535 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:14:52.971534 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f43dca-4fd2-4774-b49c-4ae198d15535 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:14:53.012819 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:14:53.031909 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ff0d24-3bc1-4a5d-bd0c-c59819d0f2ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:14:53.032534 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ff0d24-3bc1-4a5d-bd0c-c59819d0f2ff admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:14:53.032818 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ff0d24-3bc1-4a5d-bd0c-c59819d0f2ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:14:53.033095 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ff0d24-3bc1-4a5d-bd0c-c59819d0f2ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:14:53.033748 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ff0d24-3bc1-4a5d-bd0c-c59819d0f2ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:14:53.035429 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ff0d24-3bc1-4a5d-bd0c-c59819d0f2ff admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:14:53.048518 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1078/2189] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:14:52 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:14:53.053745 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6011182a-1b04-41ec-9bdf-10d99bb36519 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:14:53.060117 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6011182a-1b04-41ec-9bdf-10d99bb36519 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:14:53.065384 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2673080d-975b-4a04-ae33-e79aa56179fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:14:53.066064 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2673080d-975b-4a04-ae33-e79aa56179fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:14:53.066343 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2673080d-975b-4a04-ae33-e79aa56179fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:14:53.066616 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2673080d-975b-4a04-ae33-e79aa56179fb admin admin] PATH_INFO: `/v3/projects/f04b00e4841d4f68a5dede354be864ff/users/4b255ae026fb4c519ceb54fa393a5718/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:14:53.068515 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2673080d-975b-4a04-ae33-e79aa56179fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=f04b00e4841d4f68a5dede354be864ff, user_id=4b255ae026fb4c519ceb54fa393a5718, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:14:53.071184 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2673080d-975b-4a04-ae33-e79aa56179fb admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:14:53.082490 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1112/2190] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:14:53 2022] PUT /identity/v3/projects/f04b00e4841d4f68a5dede354be864ff/users/4b255ae026fb4c519ceb54fa393a5718/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:14:53.094117 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7057f18-3171-4141-a1aa-f5ca4ce4007b None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:14:53.094401 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7057f18-3171-4141-a1aa-f5ca4ce4007b None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:14:53.094793 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7057f18-3171-4141-a1aa-f5ca4ce4007b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:14:53.122590 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-c7057f18-3171-4141-a1aa-f5ca4ce4007b None None] MFA Rules not processed for user `4b255ae026fb4c519ceb54fa393a5718`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:14:53.124392 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-c7057f18-3171-4141-a1aa-f5ca4ce4007b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:14:53.193444 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1079/2191] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:14:53 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 03:14:53.204960 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2673080d-975b-4a04-ae33-e79aa56179fb admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:14:53.206747 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-2673080d-975b-4a04-ae33-e79aa56179fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:14:53.268815 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2673080d-975b-4a04-ae33-e79aa56179fb admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:14:53.272586 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048ee8d8-b80b-4002-8483-d900e5934c6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab599d98b0624a2e89090312bcab713f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:14:53.273230 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ee8d8-b80b-4002-8483-d900e5934c6c service cinder] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:14:53.273513 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ee8d8-b80b-4002-8483-d900e5934c6c service cinder] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:14:53.273817 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ee8d8-b80b-4002-8483-d900e5934c6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:14:53.283017 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:14:53.328841 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048ee8d8-b80b-4002-8483-d900e5934c6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:14:53.330366 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048ee8d8-b80b-4002-8483-d900e5934c6c service cinder] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:14:53.340285 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1113/2192] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:14:53 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:15:03.242128 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=47 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:15:03.456128 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=22 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:15:06.967743 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7057f18-3171-4141-a1aa-f5ca4ce4007b None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:06.977942 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7057f18-3171-4141-a1aa-f5ca4ce4007b None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:06.981494 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551edeb5-aedd-45ad-a337-5b6c608c4a40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab599d98b0624a2e89090312bcab713f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:06.982182 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551edeb5-aedd-45ad-a337-5b6c608c4a40 service cinder] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:06.982468 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551edeb5-aedd-45ad-a337-5b6c608c4a40 service cinder] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:06.982742 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551edeb5-aedd-45ad-a337-5b6c608c4a40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:06.991796 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551edeb5-aedd-45ad-a337-5b6c608c4a40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:06.993212 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551edeb5-aedd-45ad-a337-5b6c608c4a40 service cinder] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:07.002945 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1080/2193] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:15:06 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:15:12.742061 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048ee8d8-b80b-4002-8483-d900e5934c6c service cinder] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:12.743783 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-048ee8d8-b80b-4002-8483-d900e5934c6c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:12.809229 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048ee8d8-b80b-4002-8483-d900e5934c6c service cinder] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:12.813835 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76427fc3-69a7-4353-97ef-240dba8192bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:12.814479 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76427fc3-69a7-4353-97ef-240dba8192bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:12.814756 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76427fc3-69a7-4353-97ef-240dba8192bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:12.815635 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76427fc3-69a7-4353-97ef-240dba8192bd admin admin] PATH_INFO: `/v3/users/7ceb22bf95c243b98bb517abe40ce583` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:12.816813 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76427fc3-69a7-4353-97ef-240dba8192bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ceb22bf95c243b98bb517abe40ce583)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:12.818792 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76427fc3-69a7-4353-97ef-240dba8192bd admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:13.010728 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-76427fc3-69a7-4353-97ef-240dba8192bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ceb22bf95c243b98bb517abe40ce583'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:13.011322 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-76427fc3-69a7-4353-97ef-240dba8192bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ceb22bf95c243b98bb517abe40ce583'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:13.090235 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-76427fc3-69a7-4353-97ef-240dba8192bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ceb22bf95c243b98bb517abe40ce583'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:13.090850 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-76427fc3-69a7-4353-97ef-240dba8192bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ceb22bf95c243b98bb517abe40ce583'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:13.098618 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-76427fc3-69a7-4353-97ef-240dba8192bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ceb22bf95c243b98bb517abe40ce583'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:13.116885 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1114/2194] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:15:12 2022] DELETE /identity/v3/users/7ceb22bf95c243b98bb517abe40ce583 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 03:15:13.121278 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118500fe-0821-4cf5-b3f0-c02b39e76cc8 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:13.121815 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118500fe-0821-4cf5-b3f0-c02b39e76cc8 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:13.121916 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118500fe-0821-4cf5-b3f0-c02b39e76cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:13.140482 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-118500fe-0821-4cf5-b3f0-c02b39e76cc8 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:15:13.141689 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-118500fe-0821-4cf5-b3f0-c02b39e76cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:13.199957 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1081/2195] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:15:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:15:13.209588 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76427fc3-69a7-4353-97ef-240dba8192bd admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:13.211016 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-76427fc3-69a7-4353-97ef-240dba8192bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:13.267914 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76427fc3-69a7-4353-97ef-240dba8192bd admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:13.271462 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabacb53-9a7e-4591-ae4f-068fc37f70a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:13.272057 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabacb53-9a7e-4591-ae4f-068fc37f70a8 service neutron] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:13.272339 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabacb53-9a7e-4591-ae4f-068fc37f70a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:13.272611 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabacb53-9a7e-4591-ae4f-068fc37f70a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:13.281713 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabacb53-9a7e-4591-ae4f-068fc37f70a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:13.283252 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabacb53-9a7e-4591-ae4f-068fc37f70a8 service neutron] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:13.293580 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1115/2196] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:15:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:15:13.686072 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118500fe-0821-4cf5-b3f0-c02b39e76cc8 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:13.687487 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-118500fe-0821-4cf5-b3f0-c02b39e76cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:13.711927 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118500fe-0821-4cf5-b3f0-c02b39e76cc8 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:13.717679 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fc5487-f683-4357-bb06-05a94860d2ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:13.718428 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc5487-f683-4357-bb06-05a94860d2ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:13.718769 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc5487-f683-4357-bb06-05a94860d2ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:13.719223 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc5487-f683-4357-bb06-05a94860d2ee admin admin] PATH_INFO: `/v3/projects/f454dcb9b35b4ae797d048424e1065a1` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:13.720477 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fc5487-f683-4357-bb06-05a94860d2ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=f454dcb9b35b4ae797d048424e1065a1)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:13.723165 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fc5487-f683-4357-bb06-05a94860d2ee admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:13.858136 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-33fc5487-f683-4357-bb06-05a94860d2ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f454dcb9b35b4ae797d048424e1065a1', 'request_id': 'req-33fc5487-f683-4357-bb06-05a94860d2ee'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:13.865430 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-33fc5487-f683-4357-bb06-05a94860d2ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f454dcb9b35b4ae797d048424e1065a1', 'request_id': 'req-33fc5487-f683-4357-bb06-05a94860d2ee'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:13.959216 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-33fc5487-f683-4357-bb06-05a94860d2ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f454dcb9b35b4ae797d048424e1065a1', 'request_id': 'req-33fc5487-f683-4357-bb06-05a94860d2ee'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:13.971215 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-33fc5487-f683-4357-bb06-05a94860d2ee admin admin] The token cache is being invalidate because project f454dcb9b35b4ae797d048424e1065a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:15:13.971584 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-33fc5487-f683-4357-bb06-05a94860d2ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:13.973091 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-33fc5487-f683-4357-bb06-05a94860d2ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:13.974576 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1082/2197] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:15:13 2022] DELETE /identity/v3/projects/f454dcb9b35b4ae797d048424e1065a1 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:15:13.978702 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabacb53-9a7e-4591-ae4f-068fc37f70a8 service neutron] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:13.989133 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabacb53-9a7e-4591-ae4f-068fc37f70a8 service neutron] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:14.009170 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:15:14.047634 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7620c1-ec55-4c9c-84f7-93b10fa0ca3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:14.048309 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7620c1-ec55-4c9c-84f7-93b10fa0ca3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:14.048587 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7620c1-ec55-4c9c-84f7-93b10fa0ca3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:14.048857 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7620c1-ec55-4c9c-84f7-93b10fa0ca3c admin admin] PATH_INFO: `/v3/users/2a522089e1c545eb928adb65e7d3b41f` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:14.049986 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7620c1-ec55-4c9c-84f7-93b10fa0ca3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a522089e1c545eb928adb65e7d3b41f)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:14.051935 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7620c1-ec55-4c9c-84f7-93b10fa0ca3c admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:14.205634 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-be7620c1-ec55-4c9c-84f7-93b10fa0ca3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a522089e1c545eb928adb65e7d3b41f'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:14.206233 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-be7620c1-ec55-4c9c-84f7-93b10fa0ca3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a522089e1c545eb928adb65e7d3b41f'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:14.278571 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-be7620c1-ec55-4c9c-84f7-93b10fa0ca3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a522089e1c545eb928adb65e7d3b41f'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:14.279258 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-be7620c1-ec55-4c9c-84f7-93b10fa0ca3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a522089e1c545eb928adb65e7d3b41f'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:14.286616 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-be7620c1-ec55-4c9c-84f7-93b10fa0ca3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a522089e1c545eb928adb65e7d3b41f'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:14.304249 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1116/2198] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:15:13 2022] DELETE /identity/v3/users/2a522089e1c545eb928adb65e7d3b41f => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:15:14.706161 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fc5487-f683-4357-bb06-05a94860d2ee admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:14.712544 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-33fc5487-f683-4357-bb06-05a94860d2ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:14.721648 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fc5487-f683-4357-bb06-05a94860d2ee admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:14.780538 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5f1ab5-80ee-4864-81eb-d1ea02652cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:14.781185 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5f1ab5-80ee-4864-81eb-d1ea02652cd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:14.781468 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5f1ab5-80ee-4864-81eb-d1ea02652cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:14.781745 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5f1ab5-80ee-4864-81eb-d1ea02652cd9 admin admin] PATH_INFO: `/v3/projects/a96a9826450349618792e6a179fec75d` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:14.783067 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5f1ab5-80ee-4864-81eb-d1ea02652cd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a96a9826450349618792e6a179fec75d)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:14.785502 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5f1ab5-80ee-4864-81eb-d1ea02652cd9 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:14.797447 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:15:14.901615 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-2a5f1ab5-80ee-4864-81eb-d1ea02652cd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a96a9826450349618792e6a179fec75d', 'request_id': 'req-2a5f1ab5-80ee-4864-81eb-d1ea02652cd9'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:14.907588 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-2a5f1ab5-80ee-4864-81eb-d1ea02652cd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a96a9826450349618792e6a179fec75d', 'request_id': 'req-2a5f1ab5-80ee-4864-81eb-d1ea02652cd9'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:14.976261 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-2a5f1ab5-80ee-4864-81eb-d1ea02652cd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a96a9826450349618792e6a179fec75d', 'request_id': 'req-2a5f1ab5-80ee-4864-81eb-d1ea02652cd9'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:14.987397 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-2a5f1ab5-80ee-4864-81eb-d1ea02652cd9 admin admin] The token cache is being invalidate because project a96a9826450349618792e6a179fec75d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:15:14.987686 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-2a5f1ab5-80ee-4864-81eb-d1ea02652cd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:14.988302 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-2a5f1ab5-80ee-4864-81eb-d1ea02652cd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:14.989809 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1083/2199] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:15:14 2022] DELETE /identity/v3/projects/a96a9826450349618792e6a179fec75d => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 03:15:14.994629 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d70f67-6eca-48ed-9c6e-ace26ae7735f None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:14.995133 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d70f67-6eca-48ed-9c6e-ace26ae7735f None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:14.995242 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d70f67-6eca-48ed-9c6e-ace26ae7735f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:15.017854 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-06d70f67-6eca-48ed-9c6e-ace26ae7735f None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:15:15.019551 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-06d70f67-6eca-48ed-9c6e-ace26ae7735f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:15.076901 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1117/2200] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:15:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:15:15.086509 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeeef40-2d43-454b-83a0-d04258575adb None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:15.086799 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeeef40-2d43-454b-83a0-d04258575adb None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:15.087074 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeeef40-2d43-454b-83a0-d04258575adb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:15.107234 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-8aeeef40-2d43-454b-83a0-d04258575adb None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:15:15.108421 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-8aeeef40-2d43-454b-83a0-d04258575adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:15.116475 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1084/2201] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:15:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:15:15.120533 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d70f67-6eca-48ed-9c6e-ace26ae7735f None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:15.130125 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d70f67-6eca-48ed-9c6e-ace26ae7735f None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:15.134632 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a435ed58-5e25-4ecc-acd8-3bcd21934808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:15.135223 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a435ed58-5e25-4ecc-acd8-3bcd21934808 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:15.135501 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a435ed58-5e25-4ecc-acd8-3bcd21934808 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:15.135774 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a435ed58-5e25-4ecc-acd8-3bcd21934808 admin admin] PATH_INFO: `/v3/domains` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:15.136419 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a435ed58-5e25-4ecc-acd8-3bcd21934808 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 03:15:15.136724 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a435ed58-5e25-4ecc-acd8-3bcd21934808 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:15.138349 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a435ed58-5e25-4ecc-acd8-3bcd21934808 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:15.150143 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1118/2202] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Oct 14 03:15:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 03:15:15.154324 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeeef40-2d43-454b-83a0-d04258575adb None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:15.164017 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeeef40-2d43-454b-83a0-d04258575adb None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:15.168762 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e2f6df-eed4-4095-8eff-50d3fbec0d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:15.169379 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e2f6df-eed4-4095-8eff-50d3fbec0d73 admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:15.169629 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e2f6df-eed4-4095-8eff-50d3fbec0d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:15.169953 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e2f6df-eed4-4095-8eff-50d3fbec0d73 admin admin] PATH_INFO: `/v3/projects` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:15.170587 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e2f6df-eed4-4095-8eff-50d3fbec0d73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:15.172479 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e2f6df-eed4-4095-8eff-50d3fbec0d73 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:15.286636 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1085/2203] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:15:15 2022] POST /identity/v3/projects => generated 385 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:15:15.290723 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a435ed58-5e25-4ecc-acd8-3bcd21934808 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:15.300507 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a435ed58-5e25-4ecc-acd8-3bcd21934808 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:15.326176 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:15:15.357389 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200da5c5-05c2-4feb-987a-1e87c56aefee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:15.358076 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200da5c5-05c2-4feb-987a-1e87c56aefee admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:15.358384 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200da5c5-05c2-4feb-987a-1e87c56aefee admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:15.358662 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200da5c5-05c2-4feb-987a-1e87c56aefee admin admin] PATH_INFO: `/v3/users` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:15.359302 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200da5c5-05c2-4feb-987a-1e87c56aefee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:15.361247 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200da5c5-05c2-4feb-987a-1e87c56aefee admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:15.427046 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1119/2204] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:15:15 2022] POST /identity/v3/users => generated 293 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:15:15.431259 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e2f6df-eed4-4095-8eff-50d3fbec0d73 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:15.441853 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e2f6df-eed4-4095-8eff-50d3fbec0d73 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:15.446543 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a05ed5-33d3-4d51-ba61-dbf3ee950673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:15.447131 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a05ed5-33d3-4d51-ba61-dbf3ee950673 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:15.447412 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a05ed5-33d3-4d51-ba61-dbf3ee950673 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:15.447687 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a05ed5-33d3-4d51-ba61-dbf3ee950673 admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:15.448328 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a05ed5-33d3-4d51-ba61-dbf3ee950673 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:15.449961 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a05ed5-33d3-4d51-ba61-dbf3ee950673 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:15.460317 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1086/2205] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:15:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 03:15:15.464214 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200da5c5-05c2-4feb-987a-1e87c56aefee admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:15.474730 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200da5c5-05c2-4feb-987a-1e87c56aefee admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:15.479761 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d800a77a-e58c-46b3-84b8-4c46b71f0a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:15.481875 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d800a77a-e58c-46b3-84b8-4c46b71f0a9c admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:15.481875 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d800a77a-e58c-46b3-84b8-4c46b71f0a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:15.481875 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d800a77a-e58c-46b3-84b8-4c46b71f0a9c admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:15.481875 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d800a77a-e58c-46b3-84b8-4c46b71f0a9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:15.483149 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d800a77a-e58c-46b3-84b8-4c46b71f0a9c admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:15.493180 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1120/2206] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:15:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:15:15.497403 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a05ed5-33d3-4d51-ba61-dbf3ee950673 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:15.507890 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a05ed5-33d3-4d51-ba61-dbf3ee950673 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:15.512513 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c610a5-6282-4ea6-875e-433c4b669aab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:15.513154 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c610a5-6282-4ea6-875e-433c4b669aab admin admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:15.513436 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c610a5-6282-4ea6-875e-433c4b669aab admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:15.513715 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c610a5-6282-4ea6-875e-433c4b669aab admin admin] PATH_INFO: `/v3/projects/42d62e36a1ec40a2ae9a04da612934a0/users/6bd626bcf0f14e04b0e061f34e0dd118/roles/9bd0470b544240209a420f640c79c06c` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:15.529829 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c610a5-6282-4ea6-875e-433c4b669aab admin admin] RBAC: Authorizing `identity:create_grant(project_id=42d62e36a1ec40a2ae9a04da612934a0, user_id=6bd626bcf0f14e04b0e061f34e0dd118, role_id=9bd0470b544240209a420f640c79c06c)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:15.532577 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c610a5-6282-4ea6-875e-433c4b669aab admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:15.619405 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1087/2207] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:15:15 2022] PUT /identity/v3/projects/42d62e36a1ec40a2ae9a04da612934a0/users/6bd626bcf0f14e04b0e061f34e0dd118/roles/9bd0470b544240209a420f640c79c06c => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:15:15.630117 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71903796-9332-43ed-a01e-0f44b6ac2409 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:15.630405 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71903796-9332-43ed-a01e-0f44b6ac2409 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:15.630704 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71903796-9332-43ed-a01e-0f44b6ac2409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:15.659438 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-71903796-9332-43ed-a01e-0f44b6ac2409 None None] MFA Rules not processed for user `6bd626bcf0f14e04b0e061f34e0dd118`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:15:15.660641 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-71903796-9332-43ed-a01e-0f44b6ac2409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:15.722581 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1121/2208] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:15:15 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:15:15.727126 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c610a5-6282-4ea6-875e-433c4b669aab admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:15.788026 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c610a5-6282-4ea6-875e-433c4b669aab admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:15.792792 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4deee84-9660-44b5-a181-891735a8d8b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:15.793332 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4deee84-9660-44b5-a181-891735a8d8b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:15.793616 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4deee84-9660-44b5-a181-891735a8d8b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:15.793918 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4deee84-9660-44b5-a181-891735a8d8b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:15.794562 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4deee84-9660-44b5-a181-891735a8d8b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:15.796465 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4deee84-9660-44b5-a181-891735a8d8b4 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:15.906127 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1088/2209] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:15:15 2022] POST /identity/v3/projects => generated 387 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 03:15:15.910256 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71903796-9332-43ed-a01e-0f44b6ac2409 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:15.920709 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71903796-9332-43ed-a01e-0f44b6ac2409 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:15.977623 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe562ef-faec-480b-966b-a1145637118b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:15.978259 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe562ef-faec-480b-966b-a1145637118b admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:15.978541 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe562ef-faec-480b-966b-a1145637118b admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:15.978816 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe562ef-faec-480b-966b-a1145637118b admin admin] PATH_INFO: `/v3/users` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:15.979458 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe562ef-faec-480b-966b-a1145637118b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:15.981374 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe562ef-faec-480b-966b-a1145637118b admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:16.087582 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1122/2210] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:15:15 2022] POST /identity/v3/users => generated 300 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:15:16.091821 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4deee84-9660-44b5-a181-891735a8d8b4 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:16.102421 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4deee84-9660-44b5-a181-891735a8d8b4 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:16.107053 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31621360-37f0-430b-b6c3-4b823a962e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:16.107641 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31621360-37f0-430b-b6c3-4b823a962e92 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:16.107919 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31621360-37f0-430b-b6c3-4b823a962e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:16.108192 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31621360-37f0-430b-b6c3-4b823a962e92 admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:16.108826 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31621360-37f0-430b-b6c3-4b823a962e92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:16.110522 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31621360-37f0-430b-b6c3-4b823a962e92 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:16.121449 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1089/2211] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:15:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:15:16.125419 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe562ef-faec-480b-966b-a1145637118b admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:16.135234 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe562ef-faec-480b-966b-a1145637118b admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:16.139694 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554e37f4-1fef-49bd-a08c-50a5664256cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:16.140333 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554e37f4-1fef-49bd-a08c-50a5664256cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:16.140612 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554e37f4-1fef-49bd-a08c-50a5664256cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:16.140886 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554e37f4-1fef-49bd-a08c-50a5664256cb admin admin] PATH_INFO: `/v3/projects/8cccb8d6d4084991b8ecf2becaade681/users/0429e0c4702a41a3a740bcacf60523ca/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:16.155179 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554e37f4-1fef-49bd-a08c-50a5664256cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cccb8d6d4084991b8ecf2becaade681, user_id=0429e0c4702a41a3a740bcacf60523ca, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:16.157976 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554e37f4-1fef-49bd-a08c-50a5664256cb admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:16.212884 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1123/2212] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:15:16 2022] PUT /identity/v3/projects/8cccb8d6d4084991b8ecf2becaade681/users/0429e0c4702a41a3a740bcacf60523ca/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:15:16.216795 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31621360-37f0-430b-b6c3-4b823a962e92 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:16.227225 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31621360-37f0-430b-b6c3-4b823a962e92 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:16.234875 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:15:16.282769 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5181dded-5eec-4674-8653-a5da9e532fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:16.283380 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5181dded-5eec-4674-8653-a5da9e532fe4 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:16.284162 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5181dded-5eec-4674-8653-a5da9e532fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:16.284438 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5181dded-5eec-4674-8653-a5da9e532fe4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:16.285083 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5181dded-5eec-4674-8653-a5da9e532fe4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:16.286710 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5181dded-5eec-4674-8653-a5da9e532fe4 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:16.296168 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1090/2213] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:15:16 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:15:16.301021 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554e37f4-1fef-49bd-a08c-50a5664256cb admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:16.306434 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:15:16.311649 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554e37f4-1fef-49bd-a08c-50a5664256cb admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:16.316230 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e12110e-3a7c-482f-bbe3-cafc9ee45f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:16.316870 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e12110e-3a7c-482f-bbe3-cafc9ee45f8f admin admin] REQUEST_METHOD: `PUT` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:16.317153 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e12110e-3a7c-482f-bbe3-cafc9ee45f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:16.317431 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e12110e-3a7c-482f-bbe3-cafc9ee45f8f admin admin] PATH_INFO: `/v3/projects/8cccb8d6d4084991b8ecf2becaade681/users/0429e0c4702a41a3a740bcacf60523ca/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:16.319411 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e12110e-3a7c-482f-bbe3-cafc9ee45f8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cccb8d6d4084991b8ecf2becaade681, user_id=0429e0c4702a41a3a740bcacf60523ca, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:16.322152 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e12110e-3a7c-482f-bbe3-cafc9ee45f8f admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:16.332088 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1124/2214] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:15:16 2022] PUT /identity/v3/projects/8cccb8d6d4084991b8ecf2becaade681/users/0429e0c4702a41a3a740bcacf60523ca/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:15:16.342167 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa89c57-b4a0-4724-b22c-a75ea4a6040d None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:16.342388 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa89c57-b4a0-4724-b22c-a75ea4a6040d None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:16.342664 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa89c57-b4a0-4724-b22c-a75ea4a6040d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:16.370694 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-9fa89c57-b4a0-4724-b22c-a75ea4a6040d None None] MFA Rules not processed for user `0429e0c4702a41a3a740bcacf60523ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:15:16.371896 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-9fa89c57-b4a0-4724-b22c-a75ea4a6040d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:16.436698 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1091/2215] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:15:16 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:15:16.447823 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e12110e-3a7c-482f-bbe3-cafc9ee45f8f admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:16.449530 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-8e12110e-3a7c-482f-bbe3-cafc9ee45f8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:16.506410 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e12110e-3a7c-482f-bbe3-cafc9ee45f8f admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:16.510034 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3625da62-0c62-4cb7-8c5c-4fdd5a9cf6f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab599d98b0624a2e89090312bcab713f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:16.510664 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3625da62-0c62-4cb7-8c5c-4fdd5a9cf6f3 service cinder] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:16.510940 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3625da62-0c62-4cb7-8c5c-4fdd5a9cf6f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:16.511206 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3625da62-0c62-4cb7-8c5c-4fdd5a9cf6f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:16.520365 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3625da62-0c62-4cb7-8c5c-4fdd5a9cf6f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:16.522073 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3625da62-0c62-4cb7-8c5c-4fdd5a9cf6f3 service cinder] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:16.531366 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1125/2216] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:15:16 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:15:16.562580 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa89c57-b4a0-4724-b22c-a75ea4a6040d None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:16.625937 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa89c57-b4a0-4724-b22c-a75ea4a6040d None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:16.630566 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03735178-2ecf-475e-8573-3ff7c63ef304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:16.631189 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03735178-2ecf-475e-8573-3ff7c63ef304 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:16.631465 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03735178-2ecf-475e-8573-3ff7c63ef304 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:16.631737 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03735178-2ecf-475e-8573-3ff7c63ef304 admin admin] PATH_INFO: `/v3/users/6bd626bcf0f14e04b0e061f34e0dd118` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:16.632860 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03735178-2ecf-475e-8573-3ff7c63ef304 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bd626bcf0f14e04b0e061f34e0dd118)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:16.634819 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03735178-2ecf-475e-8573-3ff7c63ef304 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:16.821382 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:15:16.832167 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-03735178-2ecf-475e-8573-3ff7c63ef304 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bd626bcf0f14e04b0e061f34e0dd118'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:16.839343 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-03735178-2ecf-475e-8573-3ff7c63ef304 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bd626bcf0f14e04b0e061f34e0dd118'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:16.851063 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-03735178-2ecf-475e-8573-3ff7c63ef304 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bd626bcf0f14e04b0e061f34e0dd118'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:16.851702 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-03735178-2ecf-475e-8573-3ff7c63ef304 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bd626bcf0f14e04b0e061f34e0dd118'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:16.924690 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-03735178-2ecf-475e-8573-3ff7c63ef304 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bd626bcf0f14e04b0e061f34e0dd118'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:16.931151 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1092/2217] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:15:16 2022] DELETE /identity/v3/users/6bd626bcf0f14e04b0e061f34e0dd118 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 03:15:16.935670 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0714c9f4-38c2-456f-a05f-78e95ff038d7 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:16.935959 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0714c9f4-38c2-456f-a05f-78e95ff038d7 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:16.936240 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0714c9f4-38c2-456f-a05f-78e95ff038d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:16.956079 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-0714c9f4-38c2-456f-a05f-78e95ff038d7 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:15:16.957329 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-0714c9f4-38c2-456f-a05f-78e95ff038d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:17.016840 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1126/2218] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:15:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:15:17.027120 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03735178-2ecf-475e-8573-3ff7c63ef304 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:17.028510 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-03735178-2ecf-475e-8573-3ff7c63ef304 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:17.086679 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03735178-2ecf-475e-8573-3ff7c63ef304 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:17.090318 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35bf3a6-0227-465d-b984-d6a668ecdc6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:17.090919 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35bf3a6-0227-465d-b984-d6a668ecdc6d service neutron] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:17.091199 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35bf3a6-0227-465d-b984-d6a668ecdc6d service neutron] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:17.091472 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35bf3a6-0227-465d-b984-d6a668ecdc6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:17.100271 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35bf3a6-0227-465d-b984-d6a668ecdc6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:17.101934 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35bf3a6-0227-465d-b984-d6a668ecdc6d service neutron] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:17.111277 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1093/2219] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:15:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:15:17.503811 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0714c9f4-38c2-456f-a05f-78e95ff038d7 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:17.505320 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-0714c9f4-38c2-456f-a05f-78e95ff038d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:17.518124 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0714c9f4-38c2-456f-a05f-78e95ff038d7 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:17.522858 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4256cad4-b5ab-4771-b649-ebf53c8b3195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:17.523696 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4256cad4-b5ab-4771-b649-ebf53c8b3195 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:17.524038 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4256cad4-b5ab-4771-b649-ebf53c8b3195 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:17.524500 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4256cad4-b5ab-4771-b649-ebf53c8b3195 admin admin] PATH_INFO: `/v3/projects/42d62e36a1ec40a2ae9a04da612934a0` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:17.525684 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4256cad4-b5ab-4771-b649-ebf53c8b3195 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42d62e36a1ec40a2ae9a04da612934a0)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:17.527647 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4256cad4-b5ab-4771-b649-ebf53c8b3195 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:17.656346 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-4256cad4-b5ab-4771-b649-ebf53c8b3195 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42d62e36a1ec40a2ae9a04da612934a0', 'request_id': 'req-4256cad4-b5ab-4771-b649-ebf53c8b3195'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:17.663036 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-4256cad4-b5ab-4771-b649-ebf53c8b3195 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42d62e36a1ec40a2ae9a04da612934a0', 'request_id': 'req-4256cad4-b5ab-4771-b649-ebf53c8b3195'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:17.721915 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-4256cad4-b5ab-4771-b649-ebf53c8b3195 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42d62e36a1ec40a2ae9a04da612934a0', 'request_id': 'req-4256cad4-b5ab-4771-b649-ebf53c8b3195'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:17.732530 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-4256cad4-b5ab-4771-b649-ebf53c8b3195 admin admin] The token cache is being invalidate because project 42d62e36a1ec40a2ae9a04da612934a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81517) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:15:17.732832 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-4256cad4-b5ab-4771-b649-ebf53c8b3195 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:17.733445 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-4256cad4-b5ab-4771-b649-ebf53c8b3195 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:17.734950 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1127/2220] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:15:17 2022] DELETE /identity/v3/projects/42d62e36a1ec40a2ae9a04da612934a0 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:15:17.739683 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35bf3a6-0227-465d-b984-d6a668ecdc6d service neutron] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:17.749459 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35bf3a6-0227-465d-b984-d6a668ecdc6d service neutron] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:17.804782 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894d34e6-721f-4b28-b459-36812ef00679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:17.805407 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894d34e6-721f-4b28-b459-36812ef00679 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:17.805690 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894d34e6-721f-4b28-b459-36812ef00679 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:17.805999 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894d34e6-721f-4b28-b459-36812ef00679 admin admin] PATH_INFO: `/v3/users/0429e0c4702a41a3a740bcacf60523ca` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:17.807123 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894d34e6-721f-4b28-b459-36812ef00679 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0429e0c4702a41a3a740bcacf60523ca)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:17.809051 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894d34e6-721f-4b28-b459-36812ef00679 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:17.992481 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-894d34e6-721f-4b28-b459-36812ef00679 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0429e0c4702a41a3a740bcacf60523ca'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:18.000245 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-894d34e6-721f-4b28-b459-36812ef00679 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0429e0c4702a41a3a740bcacf60523ca'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:18.010782 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:15:18.015621 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-894d34e6-721f-4b28-b459-36812ef00679 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0429e0c4702a41a3a740bcacf60523ca'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:18.016270 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-894d34e6-721f-4b28-b459-36812ef00679 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0429e0c4702a41a3a740bcacf60523ca'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:18.104576 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-894d34e6-721f-4b28-b459-36812ef00679 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0429e0c4702a41a3a740bcacf60523ca'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:18.110693 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1094/2221] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:15:17 2022] DELETE /identity/v3/users/0429e0c4702a41a3a740bcacf60523ca => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:15:18.506530 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4256cad4-b5ab-4771-b649-ebf53c8b3195 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:18.507962 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-4256cad4-b5ab-4771-b649-ebf53c8b3195 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:18.523237 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4256cad4-b5ab-4771-b649-ebf53c8b3195 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:18.536376 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:15:18.579786 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a261a98-1f88-4e3e-89c8-2503e14e4b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:18.580407 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a261a98-1f88-4e3e-89c8-2503e14e4b2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:18.581334 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a261a98-1f88-4e3e-89c8-2503e14e4b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:18.581605 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a261a98-1f88-4e3e-89c8-2503e14e4b2a admin admin] PATH_INFO: `/v3/projects/8cccb8d6d4084991b8ecf2becaade681` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:18.582735 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a261a98-1f88-4e3e-89c8-2503e14e4b2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8cccb8d6d4084991b8ecf2becaade681)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:18.584634 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a261a98-1f88-4e3e-89c8-2503e14e4b2a admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:18.679245 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-0a261a98-1f88-4e3e-89c8-2503e14e4b2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8cccb8d6d4084991b8ecf2becaade681', 'request_id': 'req-0a261a98-1f88-4e3e-89c8-2503e14e4b2a'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:18.685704 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-0a261a98-1f88-4e3e-89c8-2503e14e4b2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8cccb8d6d4084991b8ecf2becaade681', 'request_id': 'req-0a261a98-1f88-4e3e-89c8-2503e14e4b2a'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:18.741438 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-0a261a98-1f88-4e3e-89c8-2503e14e4b2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8cccb8d6d4084991b8ecf2becaade681', 'request_id': 'req-0a261a98-1f88-4e3e-89c8-2503e14e4b2a'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:18.751841 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-0a261a98-1f88-4e3e-89c8-2503e14e4b2a admin admin] The token cache is being invalidate because project 8cccb8d6d4084991b8ecf2becaade681 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81517) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:15:18.752055 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-0a261a98-1f88-4e3e-89c8-2503e14e4b2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:18.753316 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-0a261a98-1f88-4e3e-89c8-2503e14e4b2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:18.755050 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1128/2222] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:15:18 2022] DELETE /identity/v3/projects/8cccb8d6d4084991b8ecf2becaade681 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:15:18.759753 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e15718a-78cd-43f8-ac7d-1e27f2aa2b23 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:18.760041 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e15718a-78cd-43f8-ac7d-1e27f2aa2b23 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:18.760315 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e15718a-78cd-43f8-ac7d-1e27f2aa2b23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:18.780133 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-5e15718a-78cd-43f8-ac7d-1e27f2aa2b23 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:15:18.781342 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-5e15718a-78cd-43f8-ac7d-1e27f2aa2b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:18.841520 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1095/2223] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:15:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:15:18.850260 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f6671f-fb53-47ea-aaaf-85986751bb79 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:18.850553 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f6671f-fb53-47ea-aaaf-85986751bb79 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:18.850824 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f6671f-fb53-47ea-aaaf-85986751bb79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:18.870538 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-e8f6671f-fb53-47ea-aaaf-85986751bb79 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:15:18.871809 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-e8f6671f-fb53-47ea-aaaf-85986751bb79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:18.880022 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1129/2224] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:15:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:15:18.884185 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e15718a-78cd-43f8-ac7d-1e27f2aa2b23 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:18.893971 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e15718a-78cd-43f8-ac7d-1e27f2aa2b23 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:18.898680 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2ac16c-35c8-4759-8547-a52f56eabab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:18.899277 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2ac16c-35c8-4759-8547-a52f56eabab3 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:18.899563 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2ac16c-35c8-4759-8547-a52f56eabab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:18.899839 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2ac16c-35c8-4759-8547-a52f56eabab3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:18.900554 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2ac16c-35c8-4759-8547-a52f56eabab3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 03:15:18.900829 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2ac16c-35c8-4759-8547-a52f56eabab3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:18.902502 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2ac16c-35c8-4759-8547-a52f56eabab3 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:18.916015 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1096/2225] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Oct 14 03:15:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 03:15:18.922983 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f6671f-fb53-47ea-aaaf-85986751bb79 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:18.932706 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f6671f-fb53-47ea-aaaf-85986751bb79 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:18.937371 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac7f87e-a3b5-4886-ac16-22004c706569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:18.937994 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac7f87e-a3b5-4886-ac16-22004c706569 admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:18.938274 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac7f87e-a3b5-4886-ac16-22004c706569 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:18.938545 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac7f87e-a3b5-4886-ac16-22004c706569 admin admin] PATH_INFO: `/v3/projects` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:18.939175 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac7f87e-a3b5-4886-ac16-22004c706569 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:18.941706 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac7f87e-a3b5-4886-ac16-22004c706569 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:19.035719 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1130/2226] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:15:18 2022] POST /identity/v3/projects => generated 383 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:15:19.039352 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2ac16c-35c8-4759-8547-a52f56eabab3 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:19.050392 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2ac16c-35c8-4759-8547-a52f56eabab3 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:19.114503 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6b66da-38b9-42ae-89ae-ce8881665dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:19.115196 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6b66da-38b9-42ae-89ae-ce8881665dcf admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:19.121328 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6b66da-38b9-42ae-89ae-ce8881665dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:19.121328 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6b66da-38b9-42ae-89ae-ce8881665dcf admin admin] PATH_INFO: `/v3/users` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:19.121328 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6b66da-38b9-42ae-89ae-ce8881665dcf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:19.121328 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6b66da-38b9-42ae-89ae-ce8881665dcf admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:19.232511 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1097/2227] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:15:19 2022] POST /identity/v3/users => generated 292 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:15:19.237526 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac7f87e-a3b5-4886-ac16-22004c706569 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:19.247980 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac7f87e-a3b5-4886-ac16-22004c706569 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:19.254427 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6100f145-0b6b-4363-84c3-ad1508636b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:19.255252 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6100f145-0b6b-4363-84c3-ad1508636b0c admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:19.255557 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6100f145-0b6b-4363-84c3-ad1508636b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:19.255840 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6100f145-0b6b-4363-84c3-ad1508636b0c admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:19.256833 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6100f145-0b6b-4363-84c3-ad1508636b0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:19.263422 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6100f145-0b6b-4363-84c3-ad1508636b0c admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:19.270692 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1131/2228] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:15:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:15:19.274557 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6b66da-38b9-42ae-89ae-ce8881665dcf admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:19.288820 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6b66da-38b9-42ae-89ae-ce8881665dcf admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:19.293684 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d7279f-5922-48ac-93fa-15e48a598a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:19.294336 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d7279f-5922-48ac-93fa-15e48a598a7b admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:19.294622 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d7279f-5922-48ac-93fa-15e48a598a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:19.294899 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d7279f-5922-48ac-93fa-15e48a598a7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:19.295555 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d7279f-5922-48ac-93fa-15e48a598a7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:19.298321 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d7279f-5922-48ac-93fa-15e48a598a7b admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:19.322581 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1098/2229] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:15:19 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:15:19.326133 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6100f145-0b6b-4363-84c3-ad1508636b0c admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:19.336420 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6100f145-0b6b-4363-84c3-ad1508636b0c admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:19.340943 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc108fa-7e60-4637-8ff4-aab20281a23e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:19.341588 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc108fa-7e60-4637-8ff4-aab20281a23e admin admin] REQUEST_METHOD: `PUT` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:19.341891 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc108fa-7e60-4637-8ff4-aab20281a23e admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:19.342164 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc108fa-7e60-4637-8ff4-aab20281a23e admin admin] PATH_INFO: `/v3/projects/7357a8e3551f4e3b84fd55541de4ee0c/users/e5abc9ce76c14a608f1de8b3b057a241/roles/9bd0470b544240209a420f640c79c06c` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:19.357394 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc108fa-7e60-4637-8ff4-aab20281a23e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7357a8e3551f4e3b84fd55541de4ee0c, user_id=e5abc9ce76c14a608f1de8b3b057a241, role_id=9bd0470b544240209a420f640c79c06c)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:19.360357 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc108fa-7e60-4637-8ff4-aab20281a23e admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:19.441687 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1132/2230] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:15:19 2022] PUT /identity/v3/projects/7357a8e3551f4e3b84fd55541de4ee0c/users/e5abc9ce76c14a608f1de8b3b057a241/roles/9bd0470b544240209a420f640c79c06c => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:15:19.452022 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f575e906-255c-4c8c-871b-6aa51e70be43 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:19.452304 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f575e906-255c-4c8c-871b-6aa51e70be43 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:19.452575 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f575e906-255c-4c8c-871b-6aa51e70be43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:19.480823 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-f575e906-255c-4c8c-871b-6aa51e70be43 None None] MFA Rules not processed for user `e5abc9ce76c14a608f1de8b3b057a241`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:15:19.482059 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-f575e906-255c-4c8c-871b-6aa51e70be43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:19.510703 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:15:19.546893 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1099/2231] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:15:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:15:19.551712 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc108fa-7e60-4637-8ff4-aab20281a23e admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:19.597059 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:15:19.614307 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc108fa-7e60-4637-8ff4-aab20281a23e admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:19.619033 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2632067e-ac33-4488-99aa-7464989e057b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:19.619628 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2632067e-ac33-4488-99aa-7464989e057b admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:19.619909 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2632067e-ac33-4488-99aa-7464989e057b admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:19.620278 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2632067e-ac33-4488-99aa-7464989e057b admin admin] PATH_INFO: `/v3/projects` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:19.620900 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2632067e-ac33-4488-99aa-7464989e057b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:19.622867 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2632067e-ac33-4488-99aa-7464989e057b admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:19.717802 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1133/2232] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:15:19 2022] POST /identity/v3/projects => generated 383 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:15:19.722100 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f575e906-255c-4c8c-871b-6aa51e70be43 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:19.732818 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f575e906-255c-4c8c-871b-6aa51e70be43 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:19.788651 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e2f040-127e-42ae-8daf-03dbe0a6b016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:19.789343 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e2f040-127e-42ae-8daf-03dbe0a6b016 admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:19.789629 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e2f040-127e-42ae-8daf-03dbe0a6b016 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:19.789942 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e2f040-127e-42ae-8daf-03dbe0a6b016 admin admin] PATH_INFO: `/v3/users` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:19.790587 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e2f040-127e-42ae-8daf-03dbe0a6b016 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:19.792518 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e2f040-127e-42ae-8daf-03dbe0a6b016 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:19.899646 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1100/2233] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:15:19 2022] POST /identity/v3/users => generated 298 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:15:19.903871 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2632067e-ac33-4488-99aa-7464989e057b admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:19.914995 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2632067e-ac33-4488-99aa-7464989e057b admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:19.919585 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dbf41b-1ffb-4aaf-b053-9a02ce24824b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:19.920171 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dbf41b-1ffb-4aaf-b053-9a02ce24824b admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:19.920448 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dbf41b-1ffb-4aaf-b053-9a02ce24824b admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:19.920718 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dbf41b-1ffb-4aaf-b053-9a02ce24824b admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:19.921351 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dbf41b-1ffb-4aaf-b053-9a02ce24824b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:19.922993 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dbf41b-1ffb-4aaf-b053-9a02ce24824b admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:19.932749 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1134/2234] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:15:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:15:19.936748 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e2f040-127e-42ae-8daf-03dbe0a6b016 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:19.947292 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e2f040-127e-42ae-8daf-03dbe0a6b016 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:19.951939 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476dd866-9f49-44f8-90e8-04f16ba6dee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:19.952585 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476dd866-9f49-44f8-90e8-04f16ba6dee3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:19.952871 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476dd866-9f49-44f8-90e8-04f16ba6dee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:19.953146 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476dd866-9f49-44f8-90e8-04f16ba6dee3 admin admin] PATH_INFO: `/v3/projects/e1f7ec3b04a241148f4f0764520cb61c/users/e61443c931184f30ae90cb83edb199da/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:19.968281 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476dd866-9f49-44f8-90e8-04f16ba6dee3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1f7ec3b04a241148f4f0764520cb61c, user_id=e61443c931184f30ae90cb83edb199da, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:19.970992 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476dd866-9f49-44f8-90e8-04f16ba6dee3 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:20.062255 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1101/2235] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:15:19 2022] PUT /identity/v3/projects/e1f7ec3b04a241148f4f0764520cb61c/users/e61443c931184f30ae90cb83edb199da/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:15:20.078081 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dbf41b-1ffb-4aaf-b053-9a02ce24824b admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:20.087284 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dbf41b-1ffb-4aaf-b053-9a02ce24824b admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:20.119155 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476dd866-9f49-44f8-90e8-04f16ba6dee3 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:20.120958 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-476dd866-9f49-44f8-90e8-04f16ba6dee3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:20.145604 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2023b4f4-67ba-403d-b1e7-8c0a219e218f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:20.146247 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2023b4f4-67ba-403d-b1e7-8c0a219e218f admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:20.146526 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2023b4f4-67ba-403d-b1e7-8c0a219e218f admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:20.146820 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2023b4f4-67ba-403d-b1e7-8c0a219e218f admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:20.147615 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2023b4f4-67ba-403d-b1e7-8c0a219e218f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:20.149168 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2023b4f4-67ba-403d-b1e7-8c0a219e218f admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:20.159248 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1135/2236] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:15:20 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:15:20.163139 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2023b4f4-67ba-403d-b1e7-8c0a219e218f admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:20.172830 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2023b4f4-67ba-403d-b1e7-8c0a219e218f admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:20.177902 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d05ddf6-0a8d-4475-8c72-d5b3a74d394b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:20.178520 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d05ddf6-0a8d-4475-8c72-d5b3a74d394b admin admin] REQUEST_METHOD: `PUT` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:20.178798 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d05ddf6-0a8d-4475-8c72-d5b3a74d394b admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:20.179068 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d05ddf6-0a8d-4475-8c72-d5b3a74d394b admin admin] PATH_INFO: `/v3/projects/e1f7ec3b04a241148f4f0764520cb61c/users/e61443c931184f30ae90cb83edb199da/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:20.181028 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d05ddf6-0a8d-4475-8c72-d5b3a74d394b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1f7ec3b04a241148f4f0764520cb61c, user_id=e61443c931184f30ae90cb83edb199da, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:20.183773 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d05ddf6-0a8d-4475-8c72-d5b3a74d394b admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:20.184283 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476dd866-9f49-44f8-90e8-04f16ba6dee3 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:20.188519 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2babfd24-2199-437e-8f72-2862239e58df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:20.189122 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2babfd24-2199-437e-8f72-2862239e58df service neutron] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:20.189404 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2babfd24-2199-437e-8f72-2862239e58df service neutron] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:20.189683 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2babfd24-2199-437e-8f72-2862239e58df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:20.191966 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-2babfd24-2199-437e-8f72-2862239e58df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:20.195056 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1136/2237] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:15:20 2022] PUT /identity/v3/projects/e1f7ec3b04a241148f4f0764520cb61c/users/e61443c931184f30ae90cb83edb199da/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 03:15:20.204537 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2babfd24-2199-437e-8f72-2862239e58df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:20.205084 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2168943f-472e-42b9-8fc7-5ca216fca3db None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:20.205408 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2168943f-472e-42b9-8fc7-5ca216fca3db None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:20.205833 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2168943f-472e-42b9-8fc7-5ca216fca3db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:20.206007 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2babfd24-2199-437e-8f72-2862239e58df service neutron] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:20.216062 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1102/2238] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:15:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 03:15:20.233166 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-2168943f-472e-42b9-8fc7-5ca216fca3db None None] MFA Rules not processed for user `e61443c931184f30ae90cb83edb199da`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:15:20.234390 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-2168943f-472e-42b9-8fc7-5ca216fca3db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:20.301396 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1137/2239] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:15:20 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 03:15:20.312930 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2babfd24-2199-437e-8f72-2862239e58df service neutron] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:20.314396 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-2babfd24-2199-437e-8f72-2862239e58df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:20.375191 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:15:20.380592 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2babfd24-2199-437e-8f72-2862239e58df service neutron] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:20.437504 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef623c8-f195-4df3-a207-b82232bdad58 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab599d98b0624a2e89090312bcab713f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:20.438062 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef623c8-f195-4df3-a207-b82232bdad58 service cinder] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:20.438339 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef623c8-f195-4df3-a207-b82232bdad58 service cinder] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:20.438611 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef623c8-f195-4df3-a207-b82232bdad58 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:20.448255 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef623c8-f195-4df3-a207-b82232bdad58 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:20.449686 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef623c8-f195-4df3-a207-b82232bdad58 service cinder] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:20.459339 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1103/2240] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:15:20 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:15:20.897098 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2168943f-472e-42b9-8fc7-5ca216fca3db None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:20.899737 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-2168943f-472e-42b9-8fc7-5ca216fca3db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:20.913972 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2168943f-472e-42b9-8fc7-5ca216fca3db None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:20.917703 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d444762d-0d5f-4fc1-a6c6-c9c8f4f8406c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab599d98b0624a2e89090312bcab713f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:20.918404 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d444762d-0d5f-4fc1-a6c6-c9c8f4f8406c service cinder] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:20.918783 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d444762d-0d5f-4fc1-a6c6-c9c8f4f8406c service cinder] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:20.919150 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d444762d-0d5f-4fc1-a6c6-c9c8f4f8406c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:20.940970 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:15:20.982146 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d444762d-0d5f-4fc1-a6c6-c9c8f4f8406c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:20.983684 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d444762d-0d5f-4fc1-a6c6-c9c8f4f8406c service cinder] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:20.993638 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1138/2241] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:15:20 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:15:24.086006 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef623c8-f195-4df3-a207-b82232bdad58 service cinder] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:24.087687 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-0ef623c8-f195-4df3-a207-b82232bdad58 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:24.161457 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef623c8-f195-4df3-a207-b82232bdad58 service cinder] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:24.165990 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3451a8-fcf5-4b48-9c9a-295c08f0e5ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:24.166612 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3451a8-fcf5-4b48-9c9a-295c08f0e5ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:24.166893 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3451a8-fcf5-4b48-9c9a-295c08f0e5ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:24.167170 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3451a8-fcf5-4b48-9c9a-295c08f0e5ef admin admin] PATH_INFO: `/v3/users/ab344e29f747423fb8766f2d8fbfef0b` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:24.168277 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3451a8-fcf5-4b48-9c9a-295c08f0e5ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab344e29f747423fb8766f2d8fbfef0b)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:24.170231 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3451a8-fcf5-4b48-9c9a-295c08f0e5ef admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:24.384153 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-ea3451a8-fcf5-4b48-9c9a-295c08f0e5ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab344e29f747423fb8766f2d8fbfef0b'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:24.391105 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-ea3451a8-fcf5-4b48-9c9a-295c08f0e5ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab344e29f747423fb8766f2d8fbfef0b'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:24.403575 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-ea3451a8-fcf5-4b48-9c9a-295c08f0e5ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab344e29f747423fb8766f2d8fbfef0b'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:24.404247 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-ea3451a8-fcf5-4b48-9c9a-295c08f0e5ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab344e29f747423fb8766f2d8fbfef0b'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:24.458808 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-ea3451a8-fcf5-4b48-9c9a-295c08f0e5ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab344e29f747423fb8766f2d8fbfef0b'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:24.464781 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1104/2242] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:15:24 2022] DELETE /identity/v3/users/ab344e29f747423fb8766f2d8fbfef0b => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:15:24.753008 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d444762d-0d5f-4fc1-a6c6-c9c8f4f8406c service cinder] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:24.764695 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d444762d-0d5f-4fc1-a6c6-c9c8f4f8406c service cinder] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:24.833018 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031f82e3-b9f5-4535-8167-ab76f72b392e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:24.833654 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f82e3-b9f5-4535-8167-ab76f72b392e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:24.833977 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f82e3-b9f5-4535-8167-ab76f72b392e admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:24.834238 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f82e3-b9f5-4535-8167-ab76f72b392e admin admin] PATH_INFO: `/v3/projects/55f278e0f09c45ca8ab33cd9e102446c` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:24.835368 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031f82e3-b9f5-4535-8167-ab76f72b392e admin admin] RBAC: Authorizing `identity:delete_project(project_id=55f278e0f09c45ca8ab33cd9e102446c)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:24.837268 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031f82e3-b9f5-4535-8167-ab76f72b392e admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:24.957108 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-031f82e3-b9f5-4535-8167-ab76f72b392e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55f278e0f09c45ca8ab33cd9e102446c', 'request_id': 'req-031f82e3-b9f5-4535-8167-ab76f72b392e'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:24.963625 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-031f82e3-b9f5-4535-8167-ab76f72b392e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55f278e0f09c45ca8ab33cd9e102446c', 'request_id': 'req-031f82e3-b9f5-4535-8167-ab76f72b392e'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:25.036365 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-031f82e3-b9f5-4535-8167-ab76f72b392e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55f278e0f09c45ca8ab33cd9e102446c', 'request_id': 'req-031f82e3-b9f5-4535-8167-ab76f72b392e'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:25.048398 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-031f82e3-b9f5-4535-8167-ab76f72b392e admin admin] The token cache is being invalidate because project 55f278e0f09c45ca8ab33cd9e102446c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81517) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:15:25.048701 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-031f82e3-b9f5-4535-8167-ab76f72b392e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:25.049373 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-031f82e3-b9f5-4535-8167-ab76f72b392e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:25.051475 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1139/2243] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:15:24 2022] DELETE /identity/v3/projects/55f278e0f09c45ca8ab33cd9e102446c => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:15:25.056448 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67902df8-ee47-4cc0-9ef1-738717c83997 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:25.056729 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67902df8-ee47-4cc0-9ef1-738717c83997 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:25.057043 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67902df8-ee47-4cc0-9ef1-738717c83997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:25.087910 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-67902df8-ee47-4cc0-9ef1-738717c83997 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:15:25.089497 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-67902df8-ee47-4cc0-9ef1-738717c83997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:25.103395 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:15:25.172648 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1105/2244] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:15:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:15:25.183954 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fcf08c-7f16-422f-b81d-a7a670eb49cc None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:25.184361 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fcf08c-7f16-422f-b81d-a7a670eb49cc None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:25.184705 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fcf08c-7f16-422f-b81d-a7a670eb49cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:25.218436 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-d7fcf08c-7f16-422f-b81d-a7a670eb49cc None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:15:25.219643 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-d7fcf08c-7f16-422f-b81d-a7a670eb49cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:25.228854 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1140/2245] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:15:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:15:25.233201 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67902df8-ee47-4cc0-9ef1-738717c83997 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:25.244381 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67902df8-ee47-4cc0-9ef1-738717c83997 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:25.256714 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030e919d-4a6a-48bf-a72a-95639cbb5b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:25.257338 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030e919d-4a6a-48bf-a72a-95639cbb5b88 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:25.257620 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030e919d-4a6a-48bf-a72a-95639cbb5b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:25.257978 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030e919d-4a6a-48bf-a72a-95639cbb5b88 admin admin] PATH_INFO: `/v3/domains` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:25.258662 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030e919d-4a6a-48bf-a72a-95639cbb5b88 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 03:15:25.258972 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030e919d-4a6a-48bf-a72a-95639cbb5b88 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:25.260641 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030e919d-4a6a-48bf-a72a-95639cbb5b88 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:25.272636 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1106/2246] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Oct 14 03:15:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 03:15:25.277962 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fcf08c-7f16-422f-b81d-a7a670eb49cc None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:25.289314 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fcf08c-7f16-422f-b81d-a7a670eb49cc None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:25.294080 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c235c59f-e245-4fa9-9ff8-f570165af492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:25.294721 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c235c59f-e245-4fa9-9ff8-f570165af492 admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:25.295002 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c235c59f-e245-4fa9-9ff8-f570165af492 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:25.295276 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c235c59f-e245-4fa9-9ff8-f570165af492 admin admin] PATH_INFO: `/v3/projects` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:25.295916 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c235c59f-e245-4fa9-9ff8-f570165af492 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:25.297851 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c235c59f-e245-4fa9-9ff8-f570165af492 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:25.369076 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1141/2247] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:15:25 2022] POST /identity/v3/projects => generated 387 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:15:25.371501 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030e919d-4a6a-48bf-a72a-95639cbb5b88 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:25.387285 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030e919d-4a6a-48bf-a72a-95639cbb5b88 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:25.446946 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7fd239-bd65-4c30-8da9-8d6445d1cf39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:25.447553 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7fd239-bd65-4c30-8da9-8d6445d1cf39 admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:25.447901 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7fd239-bd65-4c30-8da9-8d6445d1cf39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:25.448181 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7fd239-bd65-4c30-8da9-8d6445d1cf39 admin admin] PATH_INFO: `/v3/users` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:25.448938 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7fd239-bd65-4c30-8da9-8d6445d1cf39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:25.450938 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7fd239-bd65-4c30-8da9-8d6445d1cf39 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:25.589821 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1107/2248] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:15:25 2022] POST /identity/v3/users => generated 294 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:15:25.592546 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c235c59f-e245-4fa9-9ff8-f570165af492 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:25.603223 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c235c59f-e245-4fa9-9ff8-f570165af492 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:25.607795 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c4942c-f668-45ed-899c-a39c1f89d584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:25.608383 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c4942c-f668-45ed-899c-a39c1f89d584 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:25.608701 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c4942c-f668-45ed-899c-a39c1f89d584 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:25.609036 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c4942c-f668-45ed-899c-a39c1f89d584 admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:25.609737 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c4942c-f668-45ed-899c-a39c1f89d584 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:25.611366 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c4942c-f668-45ed-899c-a39c1f89d584 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:25.621847 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1142/2249] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:15:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:15:25.625807 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7fd239-bd65-4c30-8da9-8d6445d1cf39 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:25.636020 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7fd239-bd65-4c30-8da9-8d6445d1cf39 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:25.640622 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102bbad5-2255-4422-874f-3d70f1cf5537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:25.641249 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102bbad5-2255-4422-874f-3d70f1cf5537 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:25.641528 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102bbad5-2255-4422-874f-3d70f1cf5537 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:25.641832 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102bbad5-2255-4422-874f-3d70f1cf5537 admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:25.642473 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102bbad5-2255-4422-874f-3d70f1cf5537 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:25.644550 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102bbad5-2255-4422-874f-3d70f1cf5537 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:25.654986 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1108/2250] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:15:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:15:25.659180 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c4942c-f668-45ed-899c-a39c1f89d584 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:25.669818 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c4942c-f668-45ed-899c-a39c1f89d584 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:25.674265 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018156c3-5075-42af-8019-3f30cffae6ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:25.674901 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018156c3-5075-42af-8019-3f30cffae6ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:25.675199 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018156c3-5075-42af-8019-3f30cffae6ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:25.675470 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018156c3-5075-42af-8019-3f30cffae6ca admin admin] PATH_INFO: `/v3/projects/67b46b98051b42d9ae4b09a289d963f5/users/b36f2a1cc0124af19d3e4929d4fd8709/roles/9bd0470b544240209a420f640c79c06c` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:25.686280 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:15:25.690097 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018156c3-5075-42af-8019-3f30cffae6ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=67b46b98051b42d9ae4b09a289d963f5, user_id=b36f2a1cc0124af19d3e4929d4fd8709, role_id=9bd0470b544240209a420f640c79c06c)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:25.692724 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018156c3-5075-42af-8019-3f30cffae6ca admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:25.767074 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1143/2251] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:15:25 2022] PUT /identity/v3/projects/67b46b98051b42d9ae4b09a289d963f5/users/b36f2a1cc0124af19d3e4929d4fd8709/roles/9bd0470b544240209a420f640c79c06c => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:15:25.778969 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6a0426-2532-49df-969d-76c32753ea8a None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:25.779257 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6a0426-2532-49df-969d-76c32753ea8a None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:25.779539 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6a0426-2532-49df-969d-76c32753ea8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:25.806587 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-fe6a0426-2532-49df-969d-76c32753ea8a None None] MFA Rules not processed for user `b36f2a1cc0124af19d3e4929d4fd8709`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:15:25.807796 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-fe6a0426-2532-49df-969d-76c32753ea8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:25.870030 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1109/2252] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:15:25 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:15:25.874642 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018156c3-5075-42af-8019-3f30cffae6ca admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:25.935921 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018156c3-5075-42af-8019-3f30cffae6ca admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:25.941161 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5663c5a8-d552-4645-ad80-2441f10e1ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:25.941849 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5663c5a8-d552-4645-ad80-2441f10e1ebd admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:25.942124 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5663c5a8-d552-4645-ad80-2441f10e1ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:25.942402 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5663c5a8-d552-4645-ad80-2441f10e1ebd admin admin] PATH_INFO: `/v3/projects` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:25.943037 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5663c5a8-d552-4645-ad80-2441f10e1ebd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:25.945005 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5663c5a8-d552-4645-ad80-2441f10e1ebd admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:26.037743 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1144/2253] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:15:25 2022] POST /identity/v3/projects => generated 387 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:15:26.041821 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6a0426-2532-49df-969d-76c32753ea8a None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:26.052480 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6a0426-2532-49df-969d-76c32753ea8a None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:26.064321 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:15:26.108553 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922c4338-4167-4638-9ce8-c6201eee32b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:26.109232 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922c4338-4167-4638-9ce8-c6201eee32b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:26.109462 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922c4338-4167-4638-9ce8-c6201eee32b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:26.109735 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922c4338-4167-4638-9ce8-c6201eee32b8 admin admin] PATH_INFO: `/v3/users` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:26.110387 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922c4338-4167-4638-9ce8-c6201eee32b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:26.113097 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922c4338-4167-4638-9ce8-c6201eee32b8 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:26.202085 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1110/2254] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:15:26 2022] POST /identity/v3/users => generated 300 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:15:26.206054 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5663c5a8-d552-4645-ad80-2441f10e1ebd admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:26.216406 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5663c5a8-d552-4645-ad80-2441f10e1ebd admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:26.221144 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0316d85c-8cd6-4f4c-86c0-0489fa9aac78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:26.221737 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0316d85c-8cd6-4f4c-86c0-0489fa9aac78 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:26.222064 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0316d85c-8cd6-4f4c-86c0-0489fa9aac78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:26.222337 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0316d85c-8cd6-4f4c-86c0-0489fa9aac78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:26.222976 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0316d85c-8cd6-4f4c-86c0-0489fa9aac78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:26.224572 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0316d85c-8cd6-4f4c-86c0-0489fa9aac78 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:26.234956 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1145/2255] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:15:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:15:26.239837 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922c4338-4167-4638-9ce8-c6201eee32b8 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:26.249720 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922c4338-4167-4638-9ce8-c6201eee32b8 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:26.255033 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064bad0d-9466-46ba-955c-487ea139af2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:26.255721 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064bad0d-9466-46ba-955c-487ea139af2f admin admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:26.256007 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064bad0d-9466-46ba-955c-487ea139af2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:26.256285 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064bad0d-9466-46ba-955c-487ea139af2f admin admin] PATH_INFO: `/v3/projects/d02d1a2812a041d7b69cd6d92e53c0ce/users/210702505957420da4a12fecfda6588c/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:26.271905 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064bad0d-9466-46ba-955c-487ea139af2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d02d1a2812a041d7b69cd6d92e53c0ce, user_id=210702505957420da4a12fecfda6588c, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:26.274801 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064bad0d-9466-46ba-955c-487ea139af2f admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:26.285466 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0316d85c-8cd6-4f4c-86c0-0489fa9aac78 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:26.287211 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-0316d85c-8cd6-4f4c-86c0-0489fa9aac78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:26.308226 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0316d85c-8cd6-4f4c-86c0-0489fa9aac78 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:26.308226 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addec049-c4eb-40a0-b73a-c82479eab3c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:26.308226 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addec049-c4eb-40a0-b73a-c82479eab3c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:26.308226 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addec049-c4eb-40a0-b73a-c82479eab3c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:26.308642 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addec049-c4eb-40a0-b73a-c82479eab3c4 admin admin] PATH_INFO: `/v3/users/e5abc9ce76c14a608f1de8b3b057a241` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:26.308642 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addec049-c4eb-40a0-b73a-c82479eab3c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5abc9ce76c14a608f1de8b3b057a241)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:26.309392 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addec049-c4eb-40a0-b73a-c82479eab3c4 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:26.356928 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1111/2256] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:15:26 2022] PUT /identity/v3/projects/d02d1a2812a041d7b69cd6d92e53c0ce/users/210702505957420da4a12fecfda6588c/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:15:26.360541 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064bad0d-9466-46ba-955c-487ea139af2f admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:26.422157 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064bad0d-9466-46ba-955c-487ea139af2f admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:26.427581 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e22405c-4b2e-42eb-8c28-1a915cc1f2e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:26.428247 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e22405c-4b2e-42eb-8c28-1a915cc1f2e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:26.428534 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e22405c-4b2e-42eb-8c28-1a915cc1f2e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:26.428810 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e22405c-4b2e-42eb-8c28-1a915cc1f2e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:26.429459 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e22405c-4b2e-42eb-8c28-1a915cc1f2e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:26.431143 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e22405c-4b2e-42eb-8c28-1a915cc1f2e5 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:26.441654 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1112/2257] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:15:26 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 03:15:26.445790 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e22405c-4b2e-42eb-8c28-1a915cc1f2e5 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:26.456026 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e22405c-4b2e-42eb-8c28-1a915cc1f2e5 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:26.460400 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-addec049-c4eb-40a0-b73a-c82479eab3c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5abc9ce76c14a608f1de8b3b057a241'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:26.461118 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-addec049-c4eb-40a0-b73a-c82479eab3c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5abc9ce76c14a608f1de8b3b057a241'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:26.461301 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f36c6d-3ba9-433c-9395-c557c78c7005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:26.461997 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f36c6d-3ba9-433c-9395-c557c78c7005 admin admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:26.462281 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f36c6d-3ba9-433c-9395-c557c78c7005 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:26.462557 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f36c6d-3ba9-433c-9395-c557c78c7005 admin admin] PATH_INFO: `/v3/projects/d02d1a2812a041d7b69cd6d92e53c0ce/users/210702505957420da4a12fecfda6588c/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:26.467094 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f36c6d-3ba9-433c-9395-c557c78c7005 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d02d1a2812a041d7b69cd6d92e53c0ce, user_id=210702505957420da4a12fecfda6588c, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:26.467635 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f36c6d-3ba9-433c-9395-c557c78c7005 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:26.478807 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1113/2258] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:15:26 2022] PUT /identity/v3/projects/d02d1a2812a041d7b69cd6d92e53c0ce/users/210702505957420da4a12fecfda6588c/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:15:26.489843 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b501cc6-1eba-4d58-a52b-52e6cf7049bb None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:26.490128 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b501cc6-1eba-4d58-a52b-52e6cf7049bb None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:26.490398 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b501cc6-1eba-4d58-a52b-52e6cf7049bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:26.513546 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-addec049-c4eb-40a0-b73a-c82479eab3c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5abc9ce76c14a608f1de8b3b057a241'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:26.514306 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-addec049-c4eb-40a0-b73a-c82479eab3c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5abc9ce76c14a608f1de8b3b057a241'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:26.524199 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-addec049-c4eb-40a0-b73a-c82479eab3c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5abc9ce76c14a608f1de8b3b057a241'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:26.525959 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-0b501cc6-1eba-4d58-a52b-52e6cf7049bb None None] MFA Rules not processed for user `210702505957420da4a12fecfda6588c`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:15:26.528170 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-0b501cc6-1eba-4d58-a52b-52e6cf7049bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:26.548803 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1146/2259] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:15:26 2022] DELETE /identity/v3/users/e5abc9ce76c14a608f1de8b3b057a241 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 03:15:26.552464 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd89e3d8-93be-4d9d-a85a-ecf8505fef1e None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:26.552826 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd89e3d8-93be-4d9d-a85a-ecf8505fef1e None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:26.553231 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd89e3d8-93be-4d9d-a85a-ecf8505fef1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:26.575224 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-cd89e3d8-93be-4d9d-a85a-ecf8505fef1e None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:15:26.576515 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-cd89e3d8-93be-4d9d-a85a-ecf8505fef1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:26.595184 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:15:26.609437 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:15:26.617154 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1114/2260] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:15:26 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 03:15:26.619224 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b501cc6-1eba-4d58-a52b-52e6cf7049bb None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:26.622560 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-0b501cc6-1eba-4d58-a52b-52e6cf7049bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:26.639808 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1147/2261] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:15:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:15:26.640712 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd89e3d8-93be-4d9d-a85a-ecf8505fef1e None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:26.642012 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-cd89e3d8-93be-4d9d-a85a-ecf8505fef1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:26.694534 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b501cc6-1eba-4d58-a52b-52e6cf7049bb None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:26.699055 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc0848a-8beb-41ca-ab28-26a8ad530705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:26.699700 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc0848a-8beb-41ca-ab28-26a8ad530705 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:26.699985 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc0848a-8beb-41ca-ab28-26a8ad530705 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:26.700264 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc0848a-8beb-41ca-ab28-26a8ad530705 admin admin] PATH_INFO: `/v3/users/00d23c554e5e464380b8f9ceb4cc0f2f` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:26.701425 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc0848a-8beb-41ca-ab28-26a8ad530705 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00d23c554e5e464380b8f9ceb4cc0f2f)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:26.703404 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc0848a-8beb-41ca-ab28-26a8ad530705 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:26.711109 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd89e3d8-93be-4d9d-a85a-ecf8505fef1e None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:26.714802 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57dfcb0-f3c0-4797-8362-145c85444730 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab599d98b0624a2e89090312bcab713f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:26.715567 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57dfcb0-f3c0-4797-8362-145c85444730 service cinder] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:26.715753 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57dfcb0-f3c0-4797-8362-145c85444730 service cinder] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:26.716038 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57dfcb0-f3c0-4797-8362-145c85444730 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:26.776785 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57dfcb0-f3c0-4797-8362-145c85444730 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:26.778258 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57dfcb0-f3c0-4797-8362-145c85444730 service cinder] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:26.788107 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1148/2262] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:15:26 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:15:26.789867 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57dfcb0-f3c0-4797-8362-145c85444730 service cinder] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:26.791198 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-c57dfcb0-f3c0-4797-8362-145c85444730 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:26.853400 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57dfcb0-f3c0-4797-8362-145c85444730 service cinder] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:26.861737 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89105ba-7237-4db3-9069-b216c8c80df6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:26.862657 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89105ba-7237-4db3-9069-b216c8c80df6 service neutron] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:26.862935 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89105ba-7237-4db3-9069-b216c8c80df6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:26.863211 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89105ba-7237-4db3-9069-b216c8c80df6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:26.872489 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89105ba-7237-4db3-9069-b216c8c80df6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:26.873953 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89105ba-7237-4db3-9069-b216c8c80df6 service neutron] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:26.883227 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1149/2263] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:15:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:15:26.884077 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89105ba-7237-4db3-9069-b216c8c80df6 service neutron] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:26.885496 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-f89105ba-7237-4db3-9069-b216c8c80df6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:26.898840 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89105ba-7237-4db3-9069-b216c8c80df6 service neutron] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:26.903079 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-fbc0848a-8beb-41ca-ab28-26a8ad530705 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00d23c554e5e464380b8f9ceb4cc0f2f'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:26.903838 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a753b9-05af-43a8-9058-8892ffcfbb4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:26.904617 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a753b9-05af-43a8-9058-8892ffcfbb4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:26.904902 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a753b9-05af-43a8-9058-8892ffcfbb4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:26.905366 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a753b9-05af-43a8-9058-8892ffcfbb4c admin admin] PATH_INFO: `/v3/users/b36f2a1cc0124af19d3e4929d4fd8709` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:26.906724 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a753b9-05af-43a8-9058-8892ffcfbb4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b36f2a1cc0124af19d3e4929d4fd8709)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:26.908703 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a753b9-05af-43a8-9058-8892ffcfbb4c admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:26.911817 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-fbc0848a-8beb-41ca-ab28-26a8ad530705 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00d23c554e5e464380b8f9ceb4cc0f2f'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:26.942252 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-fbc0848a-8beb-41ca-ab28-26a8ad530705 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00d23c554e5e464380b8f9ceb4cc0f2f'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:26.943196 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-fbc0848a-8beb-41ca-ab28-26a8ad530705 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00d23c554e5e464380b8f9ceb4cc0f2f'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:26.990619 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:15:27.022216 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-fbc0848a-8beb-41ca-ab28-26a8ad530705 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00d23c554e5e464380b8f9ceb4cc0f2f'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:27.032476 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1115/2264] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:15:26 2022] DELETE /identity/v3/users/00d23c554e5e464380b8f9ceb4cc0f2f => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:15:27.033943 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c1470c-b9cd-4985-ac4a-59c9c03a1d8d None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:27.034302 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c1470c-b9cd-4985-ac4a-59c9c03a1d8d None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:27.035196 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c1470c-b9cd-4985-ac4a-59c9c03a1d8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:27.056187 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-34c1470c-b9cd-4985-ac4a-59c9c03a1d8d None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:15:27.057637 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-34c1470c-b9cd-4985-ac4a-59c9c03a1d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:27.122845 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1116/2265] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:15:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:15:27.136620 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c1470c-b9cd-4985-ac4a-59c9c03a1d8d None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:27.138055 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-34c1470c-b9cd-4985-ac4a-59c9c03a1d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:27.146566 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:15:27.147327 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-26a753b9-05af-43a8-9058-8892ffcfbb4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b36f2a1cc0124af19d3e4929d4fd8709'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:27.148028 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-26a753b9-05af-43a8-9058-8892ffcfbb4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b36f2a1cc0124af19d3e4929d4fd8709'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:27.201458 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c1470c-b9cd-4985-ac4a-59c9c03a1d8d None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:27.205375 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fde0c5-db63-4a3e-9838-828a15bb5511 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:27.206097 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fde0c5-db63-4a3e-9838-828a15bb5511 service neutron] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:27.206933 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fde0c5-db63-4a3e-9838-828a15bb5511 service neutron] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:27.210752 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fde0c5-db63-4a3e-9838-828a15bb5511 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:27.210752 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-68fde0c5-db63-4a3e-9838-828a15bb5511 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:27.221141 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fde0c5-db63-4a3e-9838-828a15bb5511 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:27.223327 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fde0c5-db63-4a3e-9838-828a15bb5511 service neutron] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:27.230675 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-26a753b9-05af-43a8-9058-8892ffcfbb4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b36f2a1cc0124af19d3e4929d4fd8709'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:27.231300 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-26a753b9-05af-43a8-9058-8892ffcfbb4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b36f2a1cc0124af19d3e4929d4fd8709'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:27.232919 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1117/2266] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:15:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 03:15:27.239661 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-26a753b9-05af-43a8-9058-8892ffcfbb4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b36f2a1cc0124af19d3e4929d4fd8709'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:27.260189 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1150/2267] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:15:26 2022] DELETE /identity/v3/users/b36f2a1cc0124af19d3e4929d4fd8709 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:15:27.264645 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b158535-e204-402b-9f25-402a5b984b90 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:27.265182 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b158535-e204-402b-9f25-402a5b984b90 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:27.265460 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b158535-e204-402b-9f25-402a5b984b90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:27.285219 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-2b158535-e204-402b-9f25-402a5b984b90 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:15:27.286460 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-2b158535-e204-402b-9f25-402a5b984b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:27.322703 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a753b9-05af-43a8-9058-8892ffcfbb4c admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:27.324529 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-26a753b9-05af-43a8-9058-8892ffcfbb4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:27.361096 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1118/2268] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:15:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 03:15:27.370752 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b158535-e204-402b-9f25-402a5b984b90 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:27.372817 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-2b158535-e204-402b-9f25-402a5b984b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:27.413113 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a753b9-05af-43a8-9058-8892ffcfbb4c admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:27.418224 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7508d54-347e-4aa5-a4d1-7274d7ee8295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:27.418858 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7508d54-347e-4aa5-a4d1-7274d7ee8295 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:27.419139 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7508d54-347e-4aa5-a4d1-7274d7ee8295 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:27.419439 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7508d54-347e-4aa5-a4d1-7274d7ee8295 admin admin] PATH_INFO: `/v3/projects/7357a8e3551f4e3b84fd55541de4ee0c` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:27.420605 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7508d54-347e-4aa5-a4d1-7274d7ee8295 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7357a8e3551f4e3b84fd55541de4ee0c)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:27.422610 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7508d54-347e-4aa5-a4d1-7274d7ee8295 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:27.434028 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b158535-e204-402b-9f25-402a5b984b90 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:27.437778 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4183f2-9ba6-4024-be3a-0158d55dfbc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:27.438422 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4183f2-9ba6-4024-be3a-0158d55dfbc3 service neutron] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:27.438704 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4183f2-9ba6-4024-be3a-0158d55dfbc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:27.438975 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4183f2-9ba6-4024-be3a-0158d55dfbc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:27.448635 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4183f2-9ba6-4024-be3a-0158d55dfbc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:27.450225 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4183f2-9ba6-4024-be3a-0158d55dfbc3 service neutron] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:27.460349 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1119/2269] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:15:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 03:15:27.548102 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-f7508d54-347e-4aa5-a4d1-7274d7ee8295 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7357a8e3551f4e3b84fd55541de4ee0c', 'request_id': 'req-f7508d54-347e-4aa5-a4d1-7274d7ee8295'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:27.554693 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-f7508d54-347e-4aa5-a4d1-7274d7ee8295 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7357a8e3551f4e3b84fd55541de4ee0c', 'request_id': 'req-f7508d54-347e-4aa5-a4d1-7274d7ee8295'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:27.647340 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-f7508d54-347e-4aa5-a4d1-7274d7ee8295 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7357a8e3551f4e3b84fd55541de4ee0c', 'request_id': 'req-f7508d54-347e-4aa5-a4d1-7274d7ee8295'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:27.659237 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-f7508d54-347e-4aa5-a4d1-7274d7ee8295 admin admin] The token cache is being invalidate because project 7357a8e3551f4e3b84fd55541de4ee0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81517) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:15:27.659550 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-f7508d54-347e-4aa5-a4d1-7274d7ee8295 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:27.660247 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-f7508d54-347e-4aa5-a4d1-7274d7ee8295 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:27.661802 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1151/2270] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:15:27 2022] DELETE /identity/v3/projects/7357a8e3551f4e3b84fd55541de4ee0c => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 03:15:27.670884 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4183f2-9ba6-4024-be3a-0158d55dfbc3 service neutron] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:27.681519 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4183f2-9ba6-4024-be3a-0158d55dfbc3 service neutron] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:27.723246 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7508d54-347e-4aa5-a4d1-7274d7ee8295 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:27.724912 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-f7508d54-347e-4aa5-a4d1-7274d7ee8295 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:27.734528 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:15:27.751482 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224aa204-ad02-4670-9afa-93a9bb12b5e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:27.752329 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224aa204-ad02-4670-9afa-93a9bb12b5e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:27.752621 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224aa204-ad02-4670-9afa-93a9bb12b5e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:27.753023 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224aa204-ad02-4670-9afa-93a9bb12b5e4 admin admin] PATH_INFO: `/v3/users/e61443c931184f30ae90cb83edb199da` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:27.754845 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224aa204-ad02-4670-9afa-93a9bb12b5e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e61443c931184f30ae90cb83edb199da)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:27.757077 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224aa204-ad02-4670-9afa-93a9bb12b5e4 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:27.772809 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:15:27.806217 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7508d54-347e-4aa5-a4d1-7274d7ee8295 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:27.810953 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1358d4ba-da64-499e-809e-7cb2495782a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:27.811583 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358d4ba-da64-499e-809e-7cb2495782a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:27.811996 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358d4ba-da64-499e-809e-7cb2495782a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:27.812278 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358d4ba-da64-499e-809e-7cb2495782a3 admin admin] PATH_INFO: `/v3/projects/1270f42959814dd0af340a1df0dee17c` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:27.813410 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1358d4ba-da64-499e-809e-7cb2495782a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1270f42959814dd0af340a1df0dee17c)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:27.815378 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1358d4ba-da64-499e-809e-7cb2495782a3 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:27.967322 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-1358d4ba-da64-499e-809e-7cb2495782a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1270f42959814dd0af340a1df0dee17c', 'request_id': 'req-1358d4ba-da64-499e-809e-7cb2495782a3'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:27.967593 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-224aa204-ad02-4670-9afa-93a9bb12b5e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e61443c931184f30ae90cb83edb199da'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:27.974323 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-1358d4ba-da64-499e-809e-7cb2495782a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1270f42959814dd0af340a1df0dee17c', 'request_id': 'req-1358d4ba-da64-499e-809e-7cb2495782a3'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:27.975546 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-224aa204-ad02-4670-9afa-93a9bb12b5e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e61443c931184f30ae90cb83edb199da'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:27.988718 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-224aa204-ad02-4670-9afa-93a9bb12b5e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e61443c931184f30ae90cb83edb199da'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:27.989408 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-224aa204-ad02-4670-9afa-93a9bb12b5e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e61443c931184f30ae90cb83edb199da'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:28.051115 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-1358d4ba-da64-499e-809e-7cb2495782a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1270f42959814dd0af340a1df0dee17c', 'request_id': 'req-1358d4ba-da64-499e-809e-7cb2495782a3'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:28.061916 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-1358d4ba-da64-499e-809e-7cb2495782a3 admin admin] The token cache is being invalidate because project 1270f42959814dd0af340a1df0dee17c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81517) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:15:28.062207 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-1358d4ba-da64-499e-809e-7cb2495782a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:28.062849 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-1358d4ba-da64-499e-809e-7cb2495782a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:28.064333 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1152/2271] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:15:27 2022] DELETE /identity/v3/projects/1270f42959814dd0af340a1df0dee17c => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 03:15:28.065086 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1358d4ba-da64-499e-809e-7cb2495782a3 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:28.066473 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-1358d4ba-da64-499e-809e-7cb2495782a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:28.107813 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-224aa204-ad02-4670-9afa-93a9bb12b5e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e61443c931184f30ae90cb83edb199da'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:28.121163 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1120/2272] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:15:27 2022] DELETE /identity/v3/users/e61443c931184f30ae90cb83edb199da => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:15:28.121163 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e258bd75-c409-4f15-a535-dd6219de2a7f None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:28.121163 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e258bd75-c409-4f15-a535-dd6219de2a7f None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:28.121163 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e258bd75-c409-4f15-a535-dd6219de2a7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:28.138523 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1358d4ba-da64-499e-809e-7cb2495782a3 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:28.139721 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-e258bd75-c409-4f15-a535-dd6219de2a7f None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:15:28.143808 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-e258bd75-c409-4f15-a535-dd6219de2a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:28.199820 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441de44b-ee37-4d43-9495-9f078290569d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:28.200893 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441de44b-ee37-4d43-9495-9f078290569d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:28.201369 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441de44b-ee37-4d43-9495-9f078290569d admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:28.201885 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441de44b-ee37-4d43-9495-9f078290569d admin admin] PATH_INFO: `/v3/projects/67b46b98051b42d9ae4b09a289d963f5` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:28.203499 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441de44b-ee37-4d43-9495-9f078290569d admin admin] RBAC: Authorizing `identity:delete_project(project_id=67b46b98051b42d9ae4b09a289d963f5)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:28.205629 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441de44b-ee37-4d43-9495-9f078290569d admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:28.206589 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1121/2273] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:15:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:15:28.215717 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39598b22-6c3f-43ad-a68f-02dda8e826dc None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:28.216003 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39598b22-6c3f-43ad-a68f-02dda8e826dc None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:28.216279 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39598b22-6c3f-43ad-a68f-02dda8e826dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:28.236515 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-39598b22-6c3f-43ad-a68f-02dda8e826dc None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:15:28.238570 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-39598b22-6c3f-43ad-a68f-02dda8e826dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:28.248121 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1122/2274] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:15:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:15:28.252204 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39598b22-6c3f-43ad-a68f-02dda8e826dc None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:28.261330 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39598b22-6c3f-43ad-a68f-02dda8e826dc None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:28.265997 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dcc28d-86ea-424f-9670-9800a3b5f9a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:28.266750 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dcc28d-86ea-424f-9670-9800a3b5f9a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:28.267035 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dcc28d-86ea-424f-9670-9800a3b5f9a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:28.267308 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dcc28d-86ea-424f-9670-9800a3b5f9a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:28.267942 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dcc28d-86ea-424f-9670-9800a3b5f9a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 03:15:28.268107 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:15:28.268235 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dcc28d-86ea-424f-9670-9800a3b5f9a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:28.269912 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dcc28d-86ea-424f-9670-9800a3b5f9a1 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:28.281331 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1123/2275] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Oct 14 03:15:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 03:15:28.286032 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dcc28d-86ea-424f-9670-9800a3b5f9a1 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:28.295251 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dcc28d-86ea-424f-9670-9800a3b5f9a1 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:28.299825 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3cf02f-031d-4fca-93b4-ef3c3ff77cfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:28.300939 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3cf02f-031d-4fca-93b4-ef3c3ff77cfd admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:28.301217 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3cf02f-031d-4fca-93b4-ef3c3ff77cfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:28.301489 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3cf02f-031d-4fca-93b4-ef3c3ff77cfd admin admin] PATH_INFO: `/v3/projects` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:28.302140 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3cf02f-031d-4fca-93b4-ef3c3ff77cfd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:28.303963 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3cf02f-031d-4fca-93b4-ef3c3ff77cfd admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:28.340241 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-441de44b-ee37-4d43-9495-9f078290569d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67b46b98051b42d9ae4b09a289d963f5', 'request_id': 'req-441de44b-ee37-4d43-9495-9f078290569d'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:28.346417 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-441de44b-ee37-4d43-9495-9f078290569d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67b46b98051b42d9ae4b09a289d963f5', 'request_id': 'req-441de44b-ee37-4d43-9495-9f078290569d'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:28.413097 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1124/2276] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:15:28 2022] POST /identity/v3/projects => generated 371 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 03:15:28.417109 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3cf02f-031d-4fca-93b4-ef3c3ff77cfd admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:28.434765 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-441de44b-ee37-4d43-9495-9f078290569d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67b46b98051b42d9ae4b09a289d963f5', 'request_id': 'req-441de44b-ee37-4d43-9495-9f078290569d'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:28.447982 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-441de44b-ee37-4d43-9495-9f078290569d admin admin] The token cache is being invalidate because project 67b46b98051b42d9ae4b09a289d963f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81517) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:15:28.448519 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-441de44b-ee37-4d43-9495-9f078290569d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:28.449226 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-441de44b-ee37-4d43-9495-9f078290569d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:28.450963 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:15:28.451122 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1153/2277] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:15:28 2022] DELETE /identity/v3/projects/67b46b98051b42d9ae4b09a289d963f5 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:15:28.455255 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441de44b-ee37-4d43-9495-9f078290569d admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:28.457580 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-441de44b-ee37-4d43-9495-9f078290569d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:28.486555 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3cf02f-031d-4fca-93b4-ef3c3ff77cfd admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:28.491767 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d59a6d-f9fd-41d8-ae17-c636d41379c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:28.492377 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d59a6d-f9fd-41d8-ae17-c636d41379c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:28.492658 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d59a6d-f9fd-41d8-ae17-c636d41379c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:28.492938 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d59a6d-f9fd-41d8-ae17-c636d41379c5 admin admin] PATH_INFO: `/v3/users` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:28.493569 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d59a6d-f9fd-41d8-ae17-c636d41379c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:28.495597 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d59a6d-f9fd-41d8-ae17-c636d41379c5 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:28.524503 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441de44b-ee37-4d43-9495-9f078290569d admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:28.529415 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17b312e-cfb0-4ab0-b910-d6b55b42dd56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:28.530084 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b312e-cfb0-4ab0-b910-d6b55b42dd56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:28.530371 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b312e-cfb0-4ab0-b910-d6b55b42dd56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:28.530649 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b312e-cfb0-4ab0-b910-d6b55b42dd56 admin admin] PATH_INFO: `/v3/users/210702505957420da4a12fecfda6588c` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:28.531818 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17b312e-cfb0-4ab0-b910-d6b55b42dd56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=210702505957420da4a12fecfda6588c)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:28.533913 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17b312e-cfb0-4ab0-b910-d6b55b42dd56 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:28.592271 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1125/2278] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:15:28 2022] POST /identity/v3/users => generated 286 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:15:28.593110 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d59a6d-f9fd-41d8-ae17-c636d41379c5 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:28.594867 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-c6d59a6d-f9fd-41d8-ae17-c636d41379c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:28.606961 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d59a6d-f9fd-41d8-ae17-c636d41379c5 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:28.611489 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d13a41-1829-435b-986d-262041e05a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:28.612331 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d13a41-1829-435b-986d-262041e05a57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:28.612658 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d13a41-1829-435b-986d-262041e05a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:28.612931 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d13a41-1829-435b-986d-262041e05a57 admin admin] PATH_INFO: `/v3/projects/e1f7ec3b04a241148f4f0764520cb61c` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:28.614076 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d13a41-1829-435b-986d-262041e05a57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1f7ec3b04a241148f4f0764520cb61c)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:28.615950 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d13a41-1829-435b-986d-262041e05a57 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:28.789561 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-b17b312e-cfb0-4ab0-b910-d6b55b42dd56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '210702505957420da4a12fecfda6588c'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:28.789807 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-c2d13a41-1829-435b-986d-262041e05a57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1f7ec3b04a241148f4f0764520cb61c', 'request_id': 'req-c2d13a41-1829-435b-986d-262041e05a57'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:28.790432 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-b17b312e-cfb0-4ab0-b910-d6b55b42dd56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '210702505957420da4a12fecfda6588c'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:28.796313 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-c2d13a41-1829-435b-986d-262041e05a57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1f7ec3b04a241148f4f0764520cb61c', 'request_id': 'req-c2d13a41-1829-435b-986d-262041e05a57'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:28.884607 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-b17b312e-cfb0-4ab0-b910-d6b55b42dd56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '210702505957420da4a12fecfda6588c'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:28.885226 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-b17b312e-cfb0-4ab0-b910-d6b55b42dd56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '210702505957420da4a12fecfda6588c'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:28.892819 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-b17b312e-cfb0-4ab0-b910-d6b55b42dd56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '210702505957420da4a12fecfda6588c'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:28.910152 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1154/2279] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:15:28 2022] DELETE /identity/v3/users/210702505957420da4a12fecfda6588c => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:15:28.910931 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17b312e-cfb0-4ab0-b910-d6b55b42dd56 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:28.912309 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-b17b312e-cfb0-4ab0-b910-d6b55b42dd56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:28.948515 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-c2d13a41-1829-435b-986d-262041e05a57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1f7ec3b04a241148f4f0764520cb61c', 'request_id': 'req-c2d13a41-1829-435b-986d-262041e05a57'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:28.961708 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-c2d13a41-1829-435b-986d-262041e05a57 admin admin] The token cache is being invalidate because project e1f7ec3b04a241148f4f0764520cb61c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:15:28.962662 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-c2d13a41-1829-435b-986d-262041e05a57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:28.963054 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-c2d13a41-1829-435b-986d-262041e05a57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:28.964672 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1126/2280] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:15:28 2022] DELETE /identity/v3/projects/e1f7ec3b04a241148f4f0764520cb61c => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:15:28.969782 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da268a2-0484-4f1d-b60d-d738fbbc3df3 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:28.970285 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da268a2-0484-4f1d-b60d-d738fbbc3df3 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:28.970638 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da268a2-0484-4f1d-b60d-d738fbbc3df3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:28.981265 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17b312e-cfb0-4ab0-b910-d6b55b42dd56 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:28.986014 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55efd022-8ea0-43d4-95f4-bd0c87744ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:28.986952 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55efd022-8ea0-43d4-95f4-bd0c87744ef7 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:28.987747 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55efd022-8ea0-43d4-95f4-bd0c87744ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:28.989082 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:15:28.989605 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55efd022-8ea0-43d4-95f4-bd0c87744ef7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:28.990419 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55efd022-8ea0-43d4-95f4-bd0c87744ef7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:28.992075 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55efd022-8ea0-43d4-95f4-bd0c87744ef7 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:28.992263 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-0da268a2-0484-4f1d-b60d-d738fbbc3df3 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:15:28.993483 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-0da268a2-0484-4f1d-b60d-d738fbbc3df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:29.003610 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1155/2281] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:15:28 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:15:29.008165 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55efd022-8ea0-43d4-95f4-bd0c87744ef7 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:29.010090 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-55efd022-8ea0-43d4-95f4-bd0c87744ef7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:29.021333 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1127/2282] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:15:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:15:29.029033 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451bdf1a-feec-418a-851f-21337e42b171 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:29.029033 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451bdf1a-feec-418a-851f-21337e42b171 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:29.029033 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451bdf1a-feec-418a-851f-21337e42b171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:29.029345 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55efd022-8ea0-43d4-95f4-bd0c87744ef7 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:29.029345 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ddcc45-5e63-44dd-8064-751ba7c9f84f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:29.029345 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ddcc45-5e63-44dd-8064-751ba7c9f84f admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:29.029345 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ddcc45-5e63-44dd-8064-751ba7c9f84f admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:29.029718 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ddcc45-5e63-44dd-8064-751ba7c9f84f admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:29.030275 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ddcc45-5e63-44dd-8064-751ba7c9f84f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:29.031866 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ddcc45-5e63-44dd-8064-751ba7c9f84f admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:29.041778 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1156/2283] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:15:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 03:15:29.042879 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-451bdf1a-feec-418a-851f-21337e42b171 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:15:29.044078 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-451bdf1a-feec-418a-851f-21337e42b171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:29.045795 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ddcc45-5e63-44dd-8064-751ba7c9f84f admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:29.053139 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1128/2284] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:15:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:15:29.055603 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ddcc45-5e63-44dd-8064-751ba7c9f84f admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:29.057221 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451bdf1a-feec-418a-851f-21337e42b171 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:29.060463 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656dfb21-36b8-4783-bd9a-82aac481bfe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:29.061101 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656dfb21-36b8-4783-bd9a-82aac481bfe2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:29.061384 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656dfb21-36b8-4783-bd9a-82aac481bfe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:29.061665 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656dfb21-36b8-4783-bd9a-82aac481bfe2 admin admin] PATH_INFO: `/v3/projects/0a302ca039794ab599996fcda417a1c6/users/42efd916cb4b4980a45199d8cc9a4389/roles/9bd0470b544240209a420f640c79c06c` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:29.066970 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451bdf1a-feec-418a-851f-21337e42b171 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:29.071680 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a5e7c7-8de2-49d0-9786-a197774b82b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:29.072286 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a5e7c7-8de2-49d0-9786-a197774b82b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:29.072562 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a5e7c7-8de2-49d0-9786-a197774b82b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:29.072836 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a5e7c7-8de2-49d0-9786-a197774b82b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:29.073492 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a5e7c7-8de2-49d0-9786-a197774b82b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 03:15:29.073829 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a5e7c7-8de2-49d0-9786-a197774b82b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:29.075423 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a5e7c7-8de2-49d0-9786-a197774b82b4 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:29.077621 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656dfb21-36b8-4783-bd9a-82aac481bfe2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a302ca039794ab599996fcda417a1c6, user_id=42efd916cb4b4980a45199d8cc9a4389, role_id=9bd0470b544240209a420f640c79c06c)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:29.080427 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656dfb21-36b8-4783-bd9a-82aac481bfe2 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:29.092389 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1129/2285] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Oct 14 03:15:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 03:15:29.093439 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a5e7c7-8de2-49d0-9786-a197774b82b4 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:29.103689 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a5e7c7-8de2-49d0-9786-a197774b82b4 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:29.108383 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd47d54-427e-4a31-ad9f-f139ed8818ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:29.110165 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd47d54-427e-4a31-ad9f-f139ed8818ff admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:29.110165 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd47d54-427e-4a31-ad9f-f139ed8818ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:29.110165 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd47d54-427e-4a31-ad9f-f139ed8818ff admin admin] PATH_INFO: `/v3/projects` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:29.110441 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd47d54-427e-4a31-ad9f-f139ed8818ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:29.112899 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd47d54-427e-4a31-ad9f-f139ed8818ff admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:29.165187 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1157/2286] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:15:29 2022] PUT /identity/v3/projects/0a302ca039794ab599996fcda417a1c6/users/42efd916cb4b4980a45199d8cc9a4389/roles/9bd0470b544240209a420f640c79c06c => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 03:15:29.171193 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033df639-c1ef-4e8a-b4da-acaa0dbd869d None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:29.171488 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033df639-c1ef-4e8a-b4da-acaa0dbd869d None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:29.171763 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033df639-c1ef-4e8a-b4da-acaa0dbd869d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:29.191350 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-033df639-c1ef-4e8a-b4da-acaa0dbd869d None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:15:29.192565 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-033df639-c1ef-4e8a-b4da-acaa0dbd869d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:29.227527 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1130/2287] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:15:29 2022] POST /identity/v3/projects => generated 383 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:15:29.229024 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd47d54-427e-4a31-ad9f-f139ed8818ff admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:29.256554 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1158/2288] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:15:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 03:15:29.271875 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033df639-c1ef-4e8a-b4da-acaa0dbd869d None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:29.273306 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-033df639-c1ef-4e8a-b4da-acaa0dbd869d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:29.298056 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd47d54-427e-4a31-ad9f-f139ed8818ff admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:29.299758 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f15032-27a1-422d-b403-f03c435d451b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:29.300907 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f15032-27a1-422d-b403-f03c435d451b admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:29.301187 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f15032-27a1-422d-b403-f03c435d451b admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:29.301463 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f15032-27a1-422d-b403-f03c435d451b admin admin] PATH_INFO: `/v3/users` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:29.302124 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f15032-27a1-422d-b403-f03c435d451b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:29.304819 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f15032-27a1-422d-b403-f03c435d451b admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:29.317908 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:15:29.329546 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033df639-c1ef-4e8a-b4da-acaa0dbd869d None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:29.379235 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb54164-dc05-4c51-8b66-ab28bbf4a3b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:29.379899 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb54164-dc05-4c51-8b66-ab28bbf4a3b9 service neutron] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:29.380182 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb54164-dc05-4c51-8b66-ab28bbf4a3b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:29.380495 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb54164-dc05-4c51-8b66-ab28bbf4a3b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:29.390048 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb54164-dc05-4c51-8b66-ab28bbf4a3b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:29.391523 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb54164-dc05-4c51-8b66-ab28bbf4a3b9 service neutron] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:29.401125 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1159/2289] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:15:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:15:29.401980 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb54164-dc05-4c51-8b66-ab28bbf4a3b9 service neutron] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:29.403263 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-8fb54164-dc05-4c51-8b66-ab28bbf4a3b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:29.419111 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb54164-dc05-4c51-8b66-ab28bbf4a3b9 service neutron] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:29.424590 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1131/2290] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:15:29 2022] POST /identity/v3/users => generated 292 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 03:15:29.426726 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869c53fd-8796-4729-aea9-0bc3b5e8d5ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:29.427445 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f15032-27a1-422d-b403-f03c435d451b admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:29.427603 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869c53fd-8796-4729-aea9-0bc3b5e8d5ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:29.427920 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869c53fd-8796-4729-aea9-0bc3b5e8d5ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:29.428342 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869c53fd-8796-4729-aea9-0bc3b5e8d5ac admin admin] PATH_INFO: `/v3/projects/d02d1a2812a041d7b69cd6d92e53c0ce` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:29.429683 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869c53fd-8796-4729-aea9-0bc3b5e8d5ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=d02d1a2812a041d7b69cd6d92e53c0ce)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:29.431944 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869c53fd-8796-4729-aea9-0bc3b5e8d5ac admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:29.443636 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f15032-27a1-422d-b403-f03c435d451b admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:29.449141 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f32113b-25ab-450a-8ddf-c502dd6ed264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:29.474730 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f32113b-25ab-450a-8ddf-c502dd6ed264 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:29.474730 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f32113b-25ab-450a-8ddf-c502dd6ed264 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:29.474730 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f32113b-25ab-450a-8ddf-c502dd6ed264 admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:29.474730 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f32113b-25ab-450a-8ddf-c502dd6ed264 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:29.474730 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f32113b-25ab-450a-8ddf-c502dd6ed264 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:29.474730 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1132/2291] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:15:29 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 03:15:29.479191 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f32113b-25ab-450a-8ddf-c502dd6ed264 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:29.490627 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f32113b-25ab-450a-8ddf-c502dd6ed264 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:29.496348 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a57d5eb-18e6-4ea6-88f9-7b08f65ffaa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:29.497184 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a57d5eb-18e6-4ea6-88f9-7b08f65ffaa4 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:29.497470 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a57d5eb-18e6-4ea6-88f9-7b08f65ffaa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:29.497834 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a57d5eb-18e6-4ea6-88f9-7b08f65ffaa4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:29.498477 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a57d5eb-18e6-4ea6-88f9-7b08f65ffaa4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:29.500472 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a57d5eb-18e6-4ea6-88f9-7b08f65ffaa4 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:29.511580 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1133/2292] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:15:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:15:29.515787 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a57d5eb-18e6-4ea6-88f9-7b08f65ffaa4 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:29.526705 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a57d5eb-18e6-4ea6-88f9-7b08f65ffaa4 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:29.531364 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8370a505-ee04-4bc3-97a1-4a7a4fbf8d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:29.531976 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8370a505-ee04-4bc3-97a1-4a7a4fbf8d7a admin admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:29.532254 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8370a505-ee04-4bc3-97a1-4a7a4fbf8d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:29.532526 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8370a505-ee04-4bc3-97a1-4a7a4fbf8d7a admin admin] PATH_INFO: `/v3/projects/68d5bc3ff5d44d5f8ee2821597888c08/users/927637d7c6274d3c821814c5cb7dbea3/roles/9bd0470b544240209a420f640c79c06c` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:29.547244 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8370a505-ee04-4bc3-97a1-4a7a4fbf8d7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=68d5bc3ff5d44d5f8ee2821597888c08, user_id=927637d7c6274d3c821814c5cb7dbea3, role_id=9bd0470b544240209a420f640c79c06c)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:29.550664 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8370a505-ee04-4bc3-97a1-4a7a4fbf8d7a admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:29.621068 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1134/2293] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:15:29 2022] PUT /identity/v3/projects/68d5bc3ff5d44d5f8ee2821597888c08/users/927637d7c6274d3c821814c5cb7dbea3/roles/9bd0470b544240209a420f640c79c06c => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 03:15:29.633476 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf6a910-3d98-4e02-9074-16a4f750705c None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:29.634025 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf6a910-3d98-4e02-9074-16a4f750705c None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:29.634483 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf6a910-3d98-4e02-9074-16a4f750705c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:29.638529 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-869c53fd-8796-4729-aea9-0bc3b5e8d5ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd02d1a2812a041d7b69cd6d92e53c0ce', 'request_id': 'req-869c53fd-8796-4729-aea9-0bc3b5e8d5ac'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:29.643435 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:15:29.648082 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-869c53fd-8796-4729-aea9-0bc3b5e8d5ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd02d1a2812a041d7b69cd6d92e53c0ce', 'request_id': 'req-869c53fd-8796-4729-aea9-0bc3b5e8d5ac'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:29.665993 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-8cf6a910-3d98-4e02-9074-16a4f750705c None None] MFA Rules not processed for user `927637d7c6274d3c821814c5cb7dbea3`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:15:29.667243 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-8cf6a910-3d98-4e02-9074-16a4f750705c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:29.735303 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1135/2294] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:15:29 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 03:15:29.746794 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf6a910-3d98-4e02-9074-16a4f750705c None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:29.748130 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-8cf6a910-3d98-4e02-9074-16a4f750705c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:29.750678 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-869c53fd-8796-4729-aea9-0bc3b5e8d5ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd02d1a2812a041d7b69cd6d92e53c0ce', 'request_id': 'req-869c53fd-8796-4729-aea9-0bc3b5e8d5ac'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:29.764178 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-869c53fd-8796-4729-aea9-0bc3b5e8d5ac admin admin] The token cache is being invalidate because project d02d1a2812a041d7b69cd6d92e53c0ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81517) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:15:29.764442 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-869c53fd-8796-4729-aea9-0bc3b5e8d5ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:29.765095 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-869c53fd-8796-4729-aea9-0bc3b5e8d5ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:29.770163 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1160/2295] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:15:29 2022] DELETE /identity/v3/projects/d02d1a2812a041d7b69cd6d92e53c0ce => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:15:29.771657 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f18a1a-52ab-4dc1-bb8b-1ba424a64f3f None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:29.771851 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f18a1a-52ab-4dc1-bb8b-1ba424a64f3f None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:29.772134 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f18a1a-52ab-4dc1-bb8b-1ba424a64f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:29.794753 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-42f18a1a-52ab-4dc1-bb8b-1ba424a64f3f None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:15:29.796058 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-42f18a1a-52ab-4dc1-bb8b-1ba424a64f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:29.833592 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf6a910-3d98-4e02-9074-16a4f750705c None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:29.838752 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1838a8-aeb9-43c8-bdf1-d037c86c59f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab599d98b0624a2e89090312bcab713f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:29.839446 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1838a8-aeb9-43c8-bdf1-d037c86c59f0 service cinder] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:29.839732 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1838a8-aeb9-43c8-bdf1-d037c86c59f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:29.840008 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1838a8-aeb9-43c8-bdf1-d037c86c59f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:29.841911 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-5c1838a8-aeb9-43c8-bdf1-d037c86c59f0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:29.855447 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1838a8-aeb9-43c8-bdf1-d037c86c59f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:29.856959 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1838a8-aeb9-43c8-bdf1-d037c86c59f0 service cinder] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:29.865900 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1161/2296] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:15:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:15:29.869205 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1136/2297] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:15:29 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 03:15:29.878652 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6ab1fe-741d-493c-87c8-17b5c510451d None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:29.878977 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6ab1fe-741d-493c-87c8-17b5c510451d None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:29.879289 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6ab1fe-741d-493c-87c8-17b5c510451d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:29.899022 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-7f6ab1fe-741d-493c-87c8-17b5c510451d None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:15:29.900214 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-7f6ab1fe-741d-493c-87c8-17b5c510451d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:29.908284 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1162/2298] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:15:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 03:15:29.912983 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1838a8-aeb9-43c8-bdf1-d037c86c59f0 service cinder] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:29.925656 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1838a8-aeb9-43c8-bdf1-d037c86c59f0 service cinder] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:29.931066 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e133f3f9-b5ca-4874-a385-7ea7bb7a7c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:29.931680 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e133f3f9-b5ca-4874-a385-7ea7bb7a7c75 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:29.931955 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e133f3f9-b5ca-4874-a385-7ea7bb7a7c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:29.932232 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e133f3f9-b5ca-4874-a385-7ea7bb7a7c75 admin admin] PATH_INFO: `/v3/domains` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:29.932882 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e133f3f9-b5ca-4874-a385-7ea7bb7a7c75 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 03:15:29.933195 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e133f3f9-b5ca-4874-a385-7ea7bb7a7c75 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:29.934866 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e133f3f9-b5ca-4874-a385-7ea7bb7a7c75 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:29.947933 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1137/2299] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Oct 14 03:15:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 03:15:29.952216 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6ab1fe-741d-493c-87c8-17b5c510451d None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:29.962254 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6ab1fe-741d-493c-87c8-17b5c510451d None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:29.967081 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418a6ad9-5e8d-4df1-b790-31c073ae9f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:29.967695 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418a6ad9-5e8d-4df1-b790-31c073ae9f79 admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:29.967978 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418a6ad9-5e8d-4df1-b790-31c073ae9f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:29.968254 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418a6ad9-5e8d-4df1-b790-31c073ae9f79 admin admin] PATH_INFO: `/v3/projects` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:29.969009 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418a6ad9-5e8d-4df1-b790-31c073ae9f79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:29.970997 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418a6ad9-5e8d-4df1-b790-31c073ae9f79 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:30.046863 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1163/2300] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:15:29 2022] POST /identity/v3/projects => generated 387 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:15:30.051044 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e133f3f9-b5ca-4874-a385-7ea7bb7a7c75 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:30.061520 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e133f3f9-b5ca-4874-a385-7ea7bb7a7c75 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:30.073327 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:15:30.126237 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afce6728-918f-4dd7-a250-8cd39916864c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:30.126862 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afce6728-918f-4dd7-a250-8cd39916864c admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:30.127134 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afce6728-918f-4dd7-a250-8cd39916864c admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:30.127412 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afce6728-918f-4dd7-a250-8cd39916864c admin admin] PATH_INFO: `/v3/users` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:30.128079 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afce6728-918f-4dd7-a250-8cd39916864c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:30.130026 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afce6728-918f-4dd7-a250-8cd39916864c admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:30.196581 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1138/2301] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:15:30 2022] POST /identity/v3/users => generated 294 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 03:15:30.200814 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418a6ad9-5e8d-4df1-b790-31c073ae9f79 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:30.211985 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418a6ad9-5e8d-4df1-b790-31c073ae9f79 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:30.216714 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebc1696-017f-4186-9b2c-f22b032043a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:30.217317 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebc1696-017f-4186-9b2c-f22b032043a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:30.217605 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebc1696-017f-4186-9b2c-f22b032043a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:30.217918 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebc1696-017f-4186-9b2c-f22b032043a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:30.218571 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebc1696-017f-4186-9b2c-f22b032043a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:30.220209 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebc1696-017f-4186-9b2c-f22b032043a1 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:30.230881 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1164/2302] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:15:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:15:30.234940 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afce6728-918f-4dd7-a250-8cd39916864c admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:30.245555 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afce6728-918f-4dd7-a250-8cd39916864c admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:30.250388 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f1d986-84b8-4e32-a879-24402b078349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:30.251002 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f1d986-84b8-4e32-a879-24402b078349 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:30.251282 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f1d986-84b8-4e32-a879-24402b078349 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:30.251556 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f1d986-84b8-4e32-a879-24402b078349 admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:30.252236 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f1d986-84b8-4e32-a879-24402b078349 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:30.254463 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f1d986-84b8-4e32-a879-24402b078349 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:30.265114 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1139/2303] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:15:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:15:30.269989 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebc1696-017f-4186-9b2c-f22b032043a1 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:30.279898 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebc1696-017f-4186-9b2c-f22b032043a1 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:30.285028 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba949a6b-6731-4c34-8147-1c5d97b692f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:30.285679 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba949a6b-6731-4c34-8147-1c5d97b692f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:30.286094 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba949a6b-6731-4c34-8147-1c5d97b692f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:30.286574 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba949a6b-6731-4c34-8147-1c5d97b692f2 admin admin] PATH_INFO: `/v3/projects/8ff45c4a3a574a2b91665fbfa4221b28/users/5bc9aaf6beaf4db585dc868f87d5b3b9/roles/9bd0470b544240209a420f640c79c06c` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:30.301773 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba949a6b-6731-4c34-8147-1c5d97b692f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ff45c4a3a574a2b91665fbfa4221b28, user_id=5bc9aaf6beaf4db585dc868f87d5b3b9, role_id=9bd0470b544240209a420f640c79c06c)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:30.304613 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba949a6b-6731-4c34-8147-1c5d97b692f2 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:30.397234 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:15:30.402973 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1165/2304] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:15:30 2022] PUT /identity/v3/projects/8ff45c4a3a574a2b91665fbfa4221b28/users/5bc9aaf6beaf4db585dc868f87d5b3b9/roles/9bd0470b544240209a420f640c79c06c => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:15:30.414474 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e3de1e-f8b7-47d8-a73e-e4659b2858cd None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:30.414764 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e3de1e-f8b7-47d8-a73e-e4659b2858cd None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:30.415040 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e3de1e-f8b7-47d8-a73e-e4659b2858cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:30.447186 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-50e3de1e-f8b7-47d8-a73e-e4659b2858cd None None] MFA Rules not processed for user `5bc9aaf6beaf4db585dc868f87d5b3b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:15:30.448522 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-50e3de1e-f8b7-47d8-a73e-e4659b2858cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:30.514833 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1140/2305] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:15:30 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:15:30.520133 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba949a6b-6731-4c34-8147-1c5d97b692f2 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:30.588414 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba949a6b-6731-4c34-8147-1c5d97b692f2 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:30.593226 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d04ca0-ba84-463d-8d3d-c690633b7149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:30.593976 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d04ca0-ba84-463d-8d3d-c690633b7149 admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:30.594208 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d04ca0-ba84-463d-8d3d-c690633b7149 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:30.594479 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d04ca0-ba84-463d-8d3d-c690633b7149 admin admin] PATH_INFO: `/v3/projects` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:30.595120 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d04ca0-ba84-463d-8d3d-c690633b7149 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:30.597010 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d04ca0-ba84-463d-8d3d-c690633b7149 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:30.732010 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1166/2306] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:15:30 2022] POST /identity/v3/projects => generated 387 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:15:30.735799 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e3de1e-f8b7-47d8-a73e-e4659b2858cd None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:30.749070 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e3de1e-f8b7-47d8-a73e-e4659b2858cd None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:30.809534 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaeec58-ccf0-4fad-b574-04f5dbfb08ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:30.810280 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaeec58-ccf0-4fad-b574-04f5dbfb08ce admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:30.810633 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaeec58-ccf0-4fad-b574-04f5dbfb08ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:30.810901 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaeec58-ccf0-4fad-b574-04f5dbfb08ce admin admin] PATH_INFO: `/v3/users` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:30.811545 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaeec58-ccf0-4fad-b574-04f5dbfb08ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:30.813484 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaeec58-ccf0-4fad-b574-04f5dbfb08ce admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:30.934537 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1141/2307] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:15:30 2022] POST /identity/v3/users => generated 300 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:15:30.935841 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d04ca0-ba84-463d-8d3d-c690633b7149 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:30.946761 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d04ca0-ba84-463d-8d3d-c690633b7149 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:30.951341 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b53c85a-796e-4264-ad0d-a4799e09f1c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:30.951938 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b53c85a-796e-4264-ad0d-a4799e09f1c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:30.952221 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b53c85a-796e-4264-ad0d-a4799e09f1c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:30.952494 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b53c85a-796e-4264-ad0d-a4799e09f1c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:30.953170 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b53c85a-796e-4264-ad0d-a4799e09f1c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:30.954867 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b53c85a-796e-4264-ad0d-a4799e09f1c4 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:30.965913 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1167/2308] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:15:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:15:30.970175 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaeec58-ccf0-4fad-b574-04f5dbfb08ce admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:30.984795 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaeec58-ccf0-4fad-b574-04f5dbfb08ce admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:30.990747 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd6ee3b-b6e6-4bfb-8fe0-40c461cc8cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:30.991215 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd6ee3b-b6e6-4bfb-8fe0-40c461cc8cd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:30.991505 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd6ee3b-b6e6-4bfb-8fe0-40c461cc8cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:30.991805 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd6ee3b-b6e6-4bfb-8fe0-40c461cc8cd1 admin admin] PATH_INFO: `/v3/projects/030acbcae72e4750b1a45e1ca9ca67fd/users/7fb355cf93b1409baac0d5d6b5bd5220/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:31.008172 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd6ee3b-b6e6-4bfb-8fe0-40c461cc8cd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=030acbcae72e4750b1a45e1ca9ca67fd, user_id=7fb355cf93b1409baac0d5d6b5bd5220, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:31.010559 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd6ee3b-b6e6-4bfb-8fe0-40c461cc8cd1 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:31.016379 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:15:31.084928 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1142/2309] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:15:30 2022] PUT /identity/v3/projects/030acbcae72e4750b1a45e1ca9ca67fd/users/7fb355cf93b1409baac0d5d6b5bd5220/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:15:31.089014 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b53c85a-796e-4264-ad0d-a4799e09f1c4 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:31.099733 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b53c85a-796e-4264-ad0d-a4799e09f1c4 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:31.158371 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2425686a-9bc6-4b75-9024-671f6b65c2c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:31.158984 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2425686a-9bc6-4b75-9024-671f6b65c2c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:31.159789 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2425686a-9bc6-4b75-9024-671f6b65c2c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:31.160064 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2425686a-9bc6-4b75-9024-671f6b65c2c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:31.160737 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2425686a-9bc6-4b75-9024-671f6b65c2c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:31.162375 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2425686a-9bc6-4b75-9024-671f6b65c2c8 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:31.172146 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1168/2310] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:15:31 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:15:31.176856 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd6ee3b-b6e6-4bfb-8fe0-40c461cc8cd1 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:31.187391 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd6ee3b-b6e6-4bfb-8fe0-40c461cc8cd1 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:31.192148 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c76c76d-5bd5-4481-bac9-b54472d97b86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:31.192808 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c76c76d-5bd5-4481-bac9-b54472d97b86 admin admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:31.193088 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c76c76d-5bd5-4481-bac9-b54472d97b86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:31.193359 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c76c76d-5bd5-4481-bac9-b54472d97b86 admin admin] PATH_INFO: `/v3/projects/030acbcae72e4750b1a45e1ca9ca67fd/users/7fb355cf93b1409baac0d5d6b5bd5220/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:31.195330 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c76c76d-5bd5-4481-bac9-b54472d97b86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=030acbcae72e4750b1a45e1ca9ca67fd, user_id=7fb355cf93b1409baac0d5d6b5bd5220, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:31.198025 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c76c76d-5bd5-4481-bac9-b54472d97b86 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:31.208891 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1143/2311] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:15:31 2022] PUT /identity/v3/projects/030acbcae72e4750b1a45e1ca9ca67fd/users/7fb355cf93b1409baac0d5d6b5bd5220/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:15:31.220911 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960836e4-8f65-407e-8761-c3acb5db347a None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:31.221210 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960836e4-8f65-407e-8761-c3acb5db347a None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:31.222687 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960836e4-8f65-407e-8761-c3acb5db347a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:31.260563 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-960836e4-8f65-407e-8761-c3acb5db347a None None] MFA Rules not processed for user `7fb355cf93b1409baac0d5d6b5bd5220`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:15:31.261860 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-960836e4-8f65-407e-8761-c3acb5db347a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:31.338135 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1169/2312] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:15:31 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:15:31.350013 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c76c76d-5bd5-4481-bac9-b54472d97b86 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:31.352759 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-8c76c76d-5bd5-4481-bac9-b54472d97b86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:31.426959 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c76c76d-5bd5-4481-bac9-b54472d97b86 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:31.434539 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a233ef6b-7304-4e07-acae-162202c40136 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab599d98b0624a2e89090312bcab713f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:31.435580 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a233ef6b-7304-4e07-acae-162202c40136 service cinder] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:31.436110 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a233ef6b-7304-4e07-acae-162202c40136 service cinder] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:31.436487 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a233ef6b-7304-4e07-acae-162202c40136 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:31.449990 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a233ef6b-7304-4e07-acae-162202c40136 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:31.451536 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a233ef6b-7304-4e07-acae-162202c40136 service cinder] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:31.463199 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1144/2313] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:15:31 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:15:31.577968 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:15:32.735017 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960836e4-8f65-407e-8761-c3acb5db347a None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:32.833103 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960836e4-8f65-407e-8761-c3acb5db347a None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:32.838826 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8fe715-bfff-4177-ab2d-3c458c66846b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:32.838826 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8fe715-bfff-4177-ab2d-3c458c66846b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:32.839470 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8fe715-bfff-4177-ab2d-3c458c66846b admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:32.839757 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8fe715-bfff-4177-ab2d-3c458c66846b admin admin] PATH_INFO: `/v3/users/5bc9aaf6beaf4db585dc868f87d5b3b9` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:32.840918 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8fe715-bfff-4177-ab2d-3c458c66846b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bc9aaf6beaf4db585dc868f87d5b3b9)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:32.842986 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8fe715-bfff-4177-ab2d-3c458c66846b admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:33.038676 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-0b8fe715-bfff-4177-ab2d-3c458c66846b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bc9aaf6beaf4db585dc868f87d5b3b9'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:33.039671 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-0b8fe715-bfff-4177-ab2d-3c458c66846b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bc9aaf6beaf4db585dc868f87d5b3b9'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:33.156291 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-0b8fe715-bfff-4177-ab2d-3c458c66846b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bc9aaf6beaf4db585dc868f87d5b3b9'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:33.157490 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-0b8fe715-bfff-4177-ab2d-3c458c66846b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bc9aaf6beaf4db585dc868f87d5b3b9'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:33.166473 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-0b8fe715-bfff-4177-ab2d-3c458c66846b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bc9aaf6beaf4db585dc868f87d5b3b9'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:33.187692 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1170/2314] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:15:32 2022] DELETE /identity/v3/users/5bc9aaf6beaf4db585dc868f87d5b3b9 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:15:33.192232 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792c1cec-234d-4ef6-a131-8edd28a10690 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:33.192771 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792c1cec-234d-4ef6-a131-8edd28a10690 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:33.193182 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792c1cec-234d-4ef6-a131-8edd28a10690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:33.215909 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-792c1cec-234d-4ef6-a131-8edd28a10690 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:15:33.217484 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-792c1cec-234d-4ef6-a131-8edd28a10690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:33.311958 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1145/2315] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:15:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:15:33.325099 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8fe715-bfff-4177-ab2d-3c458c66846b admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:33.326747 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-0b8fe715-bfff-4177-ab2d-3c458c66846b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:33.456651 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8fe715-bfff-4177-ab2d-3c458c66846b admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:33.460507 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd2d67a-67d0-40ca-aaff-0f89ee5d808e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:33.461480 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd2d67a-67d0-40ca-aaff-0f89ee5d808e service neutron] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:33.461987 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd2d67a-67d0-40ca-aaff-0f89ee5d808e service neutron] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:33.462432 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd2d67a-67d0-40ca-aaff-0f89ee5d808e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:33.473869 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd2d67a-67d0-40ca-aaff-0f89ee5d808e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:33.475667 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd2d67a-67d0-40ca-aaff-0f89ee5d808e service neutron] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:33.488948 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1171/2316] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:15:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:15:34.006974 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792c1cec-234d-4ef6-a131-8edd28a10690 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:34.006974 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-792c1cec-234d-4ef6-a131-8edd28a10690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:34.033113 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792c1cec-234d-4ef6-a131-8edd28a10690 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:34.043173 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db08a3ed-b9fb-4c95-b1eb-706c82bbfd2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:34.043841 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db08a3ed-b9fb-4c95-b1eb-706c82bbfd2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:34.045037 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db08a3ed-b9fb-4c95-b1eb-706c82bbfd2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:34.045335 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db08a3ed-b9fb-4c95-b1eb-706c82bbfd2b admin admin] PATH_INFO: `/v3/projects/8ff45c4a3a574a2b91665fbfa4221b28` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:34.046973 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db08a3ed-b9fb-4c95-b1eb-706c82bbfd2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ff45c4a3a574a2b91665fbfa4221b28)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:34.064400 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db08a3ed-b9fb-4c95-b1eb-706c82bbfd2b admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:34.169987 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-db08a3ed-b9fb-4c95-b1eb-706c82bbfd2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ff45c4a3a574a2b91665fbfa4221b28', 'request_id': 'req-db08a3ed-b9fb-4c95-b1eb-706c82bbfd2b'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:34.178847 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-db08a3ed-b9fb-4c95-b1eb-706c82bbfd2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ff45c4a3a574a2b91665fbfa4221b28', 'request_id': 'req-db08a3ed-b9fb-4c95-b1eb-706c82bbfd2b'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:34.254148 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-db08a3ed-b9fb-4c95-b1eb-706c82bbfd2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ff45c4a3a574a2b91665fbfa4221b28', 'request_id': 'req-db08a3ed-b9fb-4c95-b1eb-706c82bbfd2b'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:34.291814 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-db08a3ed-b9fb-4c95-b1eb-706c82bbfd2b admin admin] The token cache is being invalidate because project 8ff45c4a3a574a2b91665fbfa4221b28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:15:34.293632 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-db08a3ed-b9fb-4c95-b1eb-706c82bbfd2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:34.294623 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-db08a3ed-b9fb-4c95-b1eb-706c82bbfd2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:34.299408 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1146/2317] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:15:33 2022] DELETE /identity/v3/projects/8ff45c4a3a574a2b91665fbfa4221b28 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:15:34.304266 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd2d67a-67d0-40ca-aaff-0f89ee5d808e service neutron] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:34.321638 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd2d67a-67d0-40ca-aaff-0f89ee5d808e service neutron] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:34.337353 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:15:34.396533 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf75ff1-9ef2-4830-b22b-0d9caa593e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:34.397302 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf75ff1-9ef2-4830-b22b-0d9caa593e70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:34.397588 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf75ff1-9ef2-4830-b22b-0d9caa593e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:34.397899 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf75ff1-9ef2-4830-b22b-0d9caa593e70 admin admin] PATH_INFO: `/v3/users/7fb355cf93b1409baac0d5d6b5bd5220` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:34.399259 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf75ff1-9ef2-4830-b22b-0d9caa593e70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fb355cf93b1409baac0d5d6b5bd5220)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:34.401408 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf75ff1-9ef2-4830-b22b-0d9caa593e70 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:34.631525 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-dcf75ff1-9ef2-4830-b22b-0d9caa593e70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fb355cf93b1409baac0d5d6b5bd5220'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:34.632422 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-dcf75ff1-9ef2-4830-b22b-0d9caa593e70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fb355cf93b1409baac0d5d6b5bd5220'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:34.707912 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-dcf75ff1-9ef2-4830-b22b-0d9caa593e70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fb355cf93b1409baac0d5d6b5bd5220'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:34.711155 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-dcf75ff1-9ef2-4830-b22b-0d9caa593e70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fb355cf93b1409baac0d5d6b5bd5220'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:34.724619 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-dcf75ff1-9ef2-4830-b22b-0d9caa593e70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fb355cf93b1409baac0d5d6b5bd5220'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:34.756721 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1172/2318] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:15:34 2022] DELETE /identity/v3/users/7fb355cf93b1409baac0d5d6b5bd5220 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:15:35.325595 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e00c5f-fff1-44d2-b022-faf14bd73ddc None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:35.326063 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e00c5f-fff1-44d2-b022-faf14bd73ddc None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:35.326176 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e00c5f-fff1-44d2-b022-faf14bd73ddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:35.336757 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:15:35.355909 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf75ff1-9ef2-4830-b22b-0d9caa593e70 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:35.360097 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-dcf75ff1-9ef2-4830-b22b-0d9caa593e70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:35.370240 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-f8e00c5f-fff1-44d2-b022-faf14bd73ddc None None] MFA Rules not processed for user `42efd916cb4b4980a45199d8cc9a4389`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:15:35.372053 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-f8e00c5f-fff1-44d2-b022-faf14bd73ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:35.453126 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1147/2319] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:15:35 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:15:35.455045 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf75ff1-9ef2-4830-b22b-0d9caa593e70 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:35.458087 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e00c5f-fff1-44d2-b022-faf14bd73ddc None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:35.459359 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-f8e00c5f-fff1-44d2-b022-faf14bd73ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:35.460512 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7064a967-eec3-4ede-9b28-cba5c2e3bafc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:35.461216 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7064a967-eec3-4ede-9b28-cba5c2e3bafc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:35.461571 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7064a967-eec3-4ede-9b28-cba5c2e3bafc admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:35.461951 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7064a967-eec3-4ede-9b28-cba5c2e3bafc admin admin] PATH_INFO: `/v3/projects/030acbcae72e4750b1a45e1ca9ca67fd` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:35.463137 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7064a967-eec3-4ede-9b28-cba5c2e3bafc admin admin] RBAC: Authorizing `identity:delete_project(project_id=030acbcae72e4750b1a45e1ca9ca67fd)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:35.465145 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7064a967-eec3-4ede-9b28-cba5c2e3bafc admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:35.473510 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e00c5f-fff1-44d2-b022-faf14bd73ddc None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:35.478168 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8964af3e-7096-4752-bf2c-8b95817362b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:35.478750 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8964af3e-7096-4752-bf2c-8b95817362b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:35.479056 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8964af3e-7096-4752-bf2c-8b95817362b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:35.479331 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8964af3e-7096-4752-bf2c-8b95817362b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:35.479967 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8964af3e-7096-4752-bf2c-8b95817362b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:35.481888 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8964af3e-7096-4752-bf2c-8b95817362b9 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:35.551300 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:15:35.559814 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-7064a967-eec3-4ede-9b28-cba5c2e3bafc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '030acbcae72e4750b1a45e1ca9ca67fd', 'request_id': 'req-7064a967-eec3-4ede-9b28-cba5c2e3bafc'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:35.567906 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-7064a967-eec3-4ede-9b28-cba5c2e3bafc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '030acbcae72e4750b1a45e1ca9ca67fd', 'request_id': 'req-7064a967-eec3-4ede-9b28-cba5c2e3bafc'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:35.574059 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1148/2320] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:15:35 2022] POST /identity/v3/projects => generated 373 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 03:15:35.578138 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8964af3e-7096-4752-bf2c-8b95817362b9 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:35.640833 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8964af3e-7096-4752-bf2c-8b95817362b9 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:35.646269 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95db912-eb08-4c93-8485-30816230ae2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:35.646876 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95db912-eb08-4c93-8485-30816230ae2f admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:35.647158 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95db912-eb08-4c93-8485-30816230ae2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:35.647432 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95db912-eb08-4c93-8485-30816230ae2f admin admin] PATH_INFO: `/v3/users` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:35.648066 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95db912-eb08-4c93-8485-30816230ae2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:35.650227 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95db912-eb08-4c93-8485-30816230ae2f admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:35.650604 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-7064a967-eec3-4ede-9b28-cba5c2e3bafc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '030acbcae72e4750b1a45e1ca9ca67fd', 'request_id': 'req-7064a967-eec3-4ede-9b28-cba5c2e3bafc'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:35.670447 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-7064a967-eec3-4ede-9b28-cba5c2e3bafc admin admin] The token cache is being invalidate because project 030acbcae72e4750b1a45e1ca9ca67fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81517) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:15:35.670834 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-7064a967-eec3-4ede-9b28-cba5c2e3bafc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:35.671561 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-7064a967-eec3-4ede-9b28-cba5c2e3bafc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:35.673227 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1173/2321] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:15:35 2022] DELETE /identity/v3/projects/030acbcae72e4750b1a45e1ca9ca67fd => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:15:35.679289 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c3fb0a-ac91-4cd9-9f17-6e57e2a3809c None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:35.679681 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c3fb0a-ac91-4cd9-9f17-6e57e2a3809c None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:35.679965 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c3fb0a-ac91-4cd9-9f17-6e57e2a3809c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:35.700203 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-43c3fb0a-ac91-4cd9-9f17-6e57e2a3809c None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:15:35.701427 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-43c3fb0a-ac91-4cd9-9f17-6e57e2a3809c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:35.710116 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1174/2322] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:15:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 03:15:35.717492 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839921ef-a35b-4351-a9a3-89429b8d678c None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:35.717905 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839921ef-a35b-4351-a9a3-89429b8d678c None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:35.718111 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839921ef-a35b-4351-a9a3-89429b8d678c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:35.734518 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1149/2323] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:15:35 2022] POST /identity/v3/users => generated 293 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:15:35.735905 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95db912-eb08-4c93-8485-30816230ae2f admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:35.737869 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-c95db912-eb08-4c93-8485-30816230ae2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:35.741331 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-839921ef-a35b-4351-a9a3-89429b8d678c None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:15:35.742552 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-839921ef-a35b-4351-a9a3-89429b8d678c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:35.750117 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95db912-eb08-4c93-8485-30816230ae2f admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:35.751878 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1175/2324] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:15:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 03:15:35.755954 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5323d7-a1e4-4a57-83f2-1dabb427812f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:35.756349 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839921ef-a35b-4351-a9a3-89429b8d678c None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:35.756632 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5323d7-a1e4-4a57-83f2-1dabb427812f admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:35.756913 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5323d7-a1e4-4a57-83f2-1dabb427812f admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:35.757187 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5323d7-a1e4-4a57-83f2-1dabb427812f admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:35.757899 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5323d7-a1e4-4a57-83f2-1dabb427812f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:35.759563 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5323d7-a1e4-4a57-83f2-1dabb427812f admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:35.765218 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839921ef-a35b-4351-a9a3-89429b8d678c None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:35.773680 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1150/2325] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:15:35 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 03:15:35.773860 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3491ad-408c-4182-8365-4c9d800e5d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:35.773860 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3491ad-408c-4182-8365-4c9d800e5d09 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:35.773860 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3491ad-408c-4182-8365-4c9d800e5d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:35.773860 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3491ad-408c-4182-8365-4c9d800e5d09 admin admin] PATH_INFO: `/v3/domains` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:35.777623 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3491ad-408c-4182-8365-4c9d800e5d09 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 03:15:35.777623 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3491ad-408c-4182-8365-4c9d800e5d09 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:35.777806 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5323d7-a1e4-4a57-83f2-1dabb427812f admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:35.777893 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3491ad-408c-4182-8365-4c9d800e5d09 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:35.784423 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5323d7-a1e4-4a57-83f2-1dabb427812f admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:35.787744 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1176/2326] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Oct 14 03:15:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 03:15:35.792328 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3491ad-408c-4182-8365-4c9d800e5d09 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:35.806319 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3491ad-408c-4182-8365-4c9d800e5d09 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:35.808761 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaf4308-a97a-4f67-bae8-cb4a8718c00d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:35.809568 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaf4308-a97a-4f67-bae8-cb4a8718c00d admin admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:35.811222 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdecc965-83ea-4665-b048-03b931832532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:35.811695 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaf4308-a97a-4f67-bae8-cb4a8718c00d admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:35.811840 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdecc965-83ea-4665-b048-03b931832532 admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:35.812102 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdecc965-83ea-4665-b048-03b931832532 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:35.812246 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaf4308-a97a-4f67-bae8-cb4a8718c00d admin admin] PATH_INFO: `/v3/projects/cedd385212eb4fb2a79a18020ecce76b/users/4b0a436168bb45aa9635df4774e275dc/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:35.812612 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdecc965-83ea-4665-b048-03b931832532 admin admin] PATH_INFO: `/v3/projects` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:35.813291 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdecc965-83ea-4665-b048-03b931832532 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:35.816219 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdecc965-83ea-4665-b048-03b931832532 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:35.837310 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaf4308-a97a-4f67-bae8-cb4a8718c00d admin admin] RBAC: Authorizing `identity:create_grant(project_id=cedd385212eb4fb2a79a18020ecce76b, user_id=4b0a436168bb45aa9635df4774e275dc, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:35.839538 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaf4308-a97a-4f67-bae8-cb4a8718c00d admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:35.937283 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1177/2327] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:15:35 2022] POST /identity/v3/projects => generated 385 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 03:15:35.938314 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdecc965-83ea-4665-b048-03b931832532 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:35.962714 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1151/2328] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:15:35 2022] PUT /identity/v3/projects/cedd385212eb4fb2a79a18020ecce76b/users/4b0a436168bb45aa9635df4774e275dc/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 03:15:35.967080 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaf4308-a97a-4f67-bae8-cb4a8718c00d admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:36.008603 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdecc965-83ea-4665-b048-03b931832532 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:36.047016 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaf4308-a97a-4f67-bae8-cb4a8718c00d admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:36.054002 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2f69f4-82f6-4e2d-bf70-ebf8da145e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:36.054678 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2f69f4-82f6-4e2d-bf70-ebf8da145e48 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:36.054967 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2f69f4-82f6-4e2d-bf70-ebf8da145e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:36.055241 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2f69f4-82f6-4e2d-bf70-ebf8da145e48 admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:36.055972 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2f69f4-82f6-4e2d-bf70-ebf8da145e48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:36.057646 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2f69f4-82f6-4e2d-bf70-ebf8da145e48 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:36.068978 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1152/2329] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:15:35 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:15:36.073963 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2f69f4-82f6-4e2d-bf70-ebf8da145e48 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:36.076261 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a36b1b-a344-4c91-be39-65d14d5a2055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:36.076866 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a36b1b-a344-4c91-be39-65d14d5a2055 admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:36.077141 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a36b1b-a344-4c91-be39-65d14d5a2055 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:36.077411 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a36b1b-a344-4c91-be39-65d14d5a2055 admin admin] PATH_INFO: `/v3/users` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:36.078076 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a36b1b-a344-4c91-be39-65d14d5a2055 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:36.079989 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a36b1b-a344-4c91-be39-65d14d5a2055 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:36.082913 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2f69f4-82f6-4e2d-bf70-ebf8da145e48 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:36.087627 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1776a45b-ee44-4630-9a6e-1d3640d6a1c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:36.088261 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1776a45b-ee44-4630-9a6e-1d3640d6a1c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:36.088541 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1776a45b-ee44-4630-9a6e-1d3640d6a1c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:36.088816 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1776a45b-ee44-4630-9a6e-1d3640d6a1c8 admin admin] PATH_INFO: `/v3/projects/cedd385212eb4fb2a79a18020ecce76b/users/4b0a436168bb45aa9635df4774e275dc/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:36.090768 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1776a45b-ee44-4630-9a6e-1d3640d6a1c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cedd385212eb4fb2a79a18020ecce76b, user_id=4b0a436168bb45aa9635df4774e275dc, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:36.093968 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1776a45b-ee44-4630-9a6e-1d3640d6a1c8 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:36.097411 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:15:36.106458 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1153/2330] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:15:36 2022] PUT /identity/v3/projects/cedd385212eb4fb2a79a18020ecce76b/users/4b0a436168bb45aa9635df4774e275dc/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:15:36.228586 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1178/2331] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:15:35 2022] POST /identity/v3/users => generated 293 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:15:36.229882 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1776a45b-ee44-4630-9a6e-1d3640d6a1c8 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:36.306604 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1776a45b-ee44-4630-9a6e-1d3640d6a1c8 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:36.311634 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c47bed1-11b7-438c-9937-b97f3eb32232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:36.312267 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c47bed1-11b7-438c-9937-b97f3eb32232 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:36.312548 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c47bed1-11b7-438c-9937-b97f3eb32232 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:36.312824 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c47bed1-11b7-438c-9937-b97f3eb32232 admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:36.313486 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c47bed1-11b7-438c-9937-b97f3eb32232 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:36.315417 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c47bed1-11b7-438c-9937-b97f3eb32232 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:36.329883 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1154/2332] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:15:36 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:15:36.331141 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a36b1b-a344-4c91-be39-65d14d5a2055 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:36.342099 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a36b1b-a344-4c91-be39-65d14d5a2055 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:36.348098 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa6ad7d-c92a-4fac-8357-c2947b223e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:36.348957 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa6ad7d-c92a-4fac-8357-c2947b223e5e admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:36.349176 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa6ad7d-c92a-4fac-8357-c2947b223e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:36.349520 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa6ad7d-c92a-4fac-8357-c2947b223e5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:36.350142 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa6ad7d-c92a-4fac-8357-c2947b223e5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:36.351862 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa6ad7d-c92a-4fac-8357-c2947b223e5e admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:36.367578 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1179/2333] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:15:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 03:15:36.368299 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c47bed1-11b7-438c-9937-b97f3eb32232 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:36.379331 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c47bed1-11b7-438c-9937-b97f3eb32232 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:36.395226 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8926b7ba-e0f0-4445-87af-40b54895bd6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:36.395226 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8926b7ba-e0f0-4445-87af-40b54895bd6e admin admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:36.395226 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8926b7ba-e0f0-4445-87af-40b54895bd6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:36.395226 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8926b7ba-e0f0-4445-87af-40b54895bd6e admin admin] PATH_INFO: `/v3/projects/dd17139d63c444beaed77f54bd4ff033/users/840e436082a44317b74b804a12a7fe90/roles/9bd0470b544240209a420f640c79c06c` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:36.403675 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8926b7ba-e0f0-4445-87af-40b54895bd6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd17139d63c444beaed77f54bd4ff033, user_id=840e436082a44317b74b804a12a7fe90, role_id=9bd0470b544240209a420f640c79c06c)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:36.406506 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8926b7ba-e0f0-4445-87af-40b54895bd6e admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:36.498018 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1155/2334] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:15:36 2022] PUT /identity/v3/projects/dd17139d63c444beaed77f54bd4ff033/users/840e436082a44317b74b804a12a7fe90/roles/9bd0470b544240209a420f640c79c06c => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:15:36.509413 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0078a193-af72-4eec-a22b-d9938022433b None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:36.509701 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0078a193-af72-4eec-a22b-d9938022433b None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:36.510018 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0078a193-af72-4eec-a22b-d9938022433b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:36.517406 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:15:36.546324 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-0078a193-af72-4eec-a22b-d9938022433b None None] MFA Rules not processed for user `840e436082a44317b74b804a12a7fe90`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:15:36.547541 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-0078a193-af72-4eec-a22b-d9938022433b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:36.612053 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1180/2335] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:15:36 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:15:36.616772 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8926b7ba-e0f0-4445-87af-40b54895bd6e admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:36.693195 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8926b7ba-e0f0-4445-87af-40b54895bd6e admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:36.699088 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0292e853-dcfb-4ad9-a197-fc231f597ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:36.699501 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0292e853-dcfb-4ad9-a197-fc231f597ef0 admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:36.699791 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0292e853-dcfb-4ad9-a197-fc231f597ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:36.700068 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0292e853-dcfb-4ad9-a197-fc231f597ef0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:36.700707 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0292e853-dcfb-4ad9-a197-fc231f597ef0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:36.702674 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0292e853-dcfb-4ad9-a197-fc231f597ef0 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:36.836046 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1156/2336] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:15:36 2022] POST /identity/v3/projects => generated 385 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:15:36.837292 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0078a193-af72-4eec-a22b-d9938022433b None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:36.848927 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0078a193-af72-4eec-a22b-d9938022433b None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:36.945424 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c9806f-2f2c-47ca-8ad0-a2c5a8b32d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:36.946185 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c9806f-2f2c-47ca-8ad0-a2c5a8b32d9b admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:36.946749 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c9806f-2f2c-47ca-8ad0-a2c5a8b32d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:36.950416 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c9806f-2f2c-47ca-8ad0-a2c5a8b32d9b admin admin] PATH_INFO: `/v3/users` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:36.950416 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c9806f-2f2c-47ca-8ad0-a2c5a8b32d9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:36.951819 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c9806f-2f2c-47ca-8ad0-a2c5a8b32d9b admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:37.046676 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1181/2337] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:15:36 2022] POST /identity/v3/users => generated 293 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:15:37.047528 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0292e853-dcfb-4ad9-a197-fc231f597ef0 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:37.054721 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:15:37.059541 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0292e853-dcfb-4ad9-a197-fc231f597ef0 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:37.066798 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8ef634-6687-4991-9dc8-a5a739f4ecf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:37.067538 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8ef634-6687-4991-9dc8-a5a739f4ecf9 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:37.067886 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8ef634-6687-4991-9dc8-a5a739f4ecf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:37.068228 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8ef634-6687-4991-9dc8-a5a739f4ecf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:37.068945 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8ef634-6687-4991-9dc8-a5a739f4ecf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:37.070890 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8ef634-6687-4991-9dc8-a5a739f4ecf9 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:37.094898 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1157/2338] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:15:37 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:15:37.095862 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c9806f-2f2c-47ca-8ad0-a2c5a8b32d9b admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:37.106858 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c9806f-2f2c-47ca-8ad0-a2c5a8b32d9b admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:37.113010 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2715d415-9db7-4d31-8c6c-8e66e97a0792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:37.113680 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2715d415-9db7-4d31-8c6c-8e66e97a0792 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:37.113974 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2715d415-9db7-4d31-8c6c-8e66e97a0792 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:37.114284 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2715d415-9db7-4d31-8c6c-8e66e97a0792 admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:37.120408 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2715d415-9db7-4d31-8c6c-8e66e97a0792 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:37.120408 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2715d415-9db7-4d31-8c6c-8e66e97a0792 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:37.127511 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1182/2339] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:15:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:15:37.132324 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8ef634-6687-4991-9dc8-a5a739f4ecf9 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:37.144365 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8ef634-6687-4991-9dc8-a5a739f4ecf9 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:37.150439 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e860da-e6fd-4edd-9038-2e4444cc5bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:37.150439 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e860da-e6fd-4edd-9038-2e4444cc5bcb admin admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:37.150439 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e860da-e6fd-4edd-9038-2e4444cc5bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:37.150439 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e860da-e6fd-4edd-9038-2e4444cc5bcb admin admin] PATH_INFO: `/v3/projects/73319bfb27454f8c93514f7cb0a6de67/users/bc5389de37cd43e383e763ccb7342dd2/roles/9bd0470b544240209a420f640c79c06c` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:37.164389 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e860da-e6fd-4edd-9038-2e4444cc5bcb admin admin] RBAC: Authorizing `identity:create_grant(project_id=73319bfb27454f8c93514f7cb0a6de67, user_id=bc5389de37cd43e383e763ccb7342dd2, role_id=9bd0470b544240209a420f640c79c06c)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:37.167114 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e860da-e6fd-4edd-9038-2e4444cc5bcb admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:37.286113 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1158/2340] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:15:37 2022] PUT /identity/v3/projects/73319bfb27454f8c93514f7cb0a6de67/users/bc5389de37cd43e383e763ccb7342dd2/roles/9bd0470b544240209a420f640c79c06c => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 03:15:37.296610 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17cbd78-e06e-434a-90a7-96fe0d0d1c8f None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:37.297012 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17cbd78-e06e-434a-90a7-96fe0d0d1c8f None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:37.297365 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17cbd78-e06e-434a-90a7-96fe0d0d1c8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:37.327276 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-c17cbd78-e06e-434a-90a7-96fe0d0d1c8f None None] MFA Rules not processed for user `bc5389de37cd43e383e763ccb7342dd2`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:15:37.328129 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-c17cbd78-e06e-434a-90a7-96fe0d0d1c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:37.398376 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:15:37.406794 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1183/2341] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:15:37 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:15:37.414251 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e860da-e6fd-4edd-9038-2e4444cc5bcb admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:37.484234 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e860da-e6fd-4edd-9038-2e4444cc5bcb admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:37.490015 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d58e6a4-7647-4323-a2aa-d235c5a7242a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:37.490644 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d58e6a4-7647-4323-a2aa-d235c5a7242a admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:37.490931 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d58e6a4-7647-4323-a2aa-d235c5a7242a admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:37.495966 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d58e6a4-7647-4323-a2aa-d235c5a7242a admin admin] PATH_INFO: `/v3/projects` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:37.495966 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d58e6a4-7647-4323-a2aa-d235c5a7242a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:37.495966 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d58e6a4-7647-4323-a2aa-d235c5a7242a admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:37.601657 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1159/2342] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:15:37 2022] POST /identity/v3/projects => generated 383 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 03:15:37.605572 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17cbd78-e06e-434a-90a7-96fe0d0d1c8f None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:37.616828 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17cbd78-e06e-434a-90a7-96fe0d0d1c8f None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:37.680729 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75c7eab-5f87-4b60-9b8d-f8f57b3e1ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:37.681926 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75c7eab-5f87-4b60-9b8d-f8f57b3e1ebc admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:37.682126 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75c7eab-5f87-4b60-9b8d-f8f57b3e1ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:37.682488 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75c7eab-5f87-4b60-9b8d-f8f57b3e1ebc admin admin] PATH_INFO: `/v3/users` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:37.683498 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75c7eab-5f87-4b60-9b8d-f8f57b3e1ebc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:37.685637 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75c7eab-5f87-4b60-9b8d-f8f57b3e1ebc admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:37.786375 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1184/2343] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:15:37 2022] POST /identity/v3/users => generated 298 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:15:37.788480 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d58e6a4-7647-4323-a2aa-d235c5a7242a admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:37.801720 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d58e6a4-7647-4323-a2aa-d235c5a7242a admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:37.809893 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716e5b09-0f43-4eb4-a7bb-0b7975cfeef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:37.809893 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716e5b09-0f43-4eb4-a7bb-0b7975cfeef7 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:37.809893 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716e5b09-0f43-4eb4-a7bb-0b7975cfeef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:37.809893 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716e5b09-0f43-4eb4-a7bb-0b7975cfeef7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:37.810257 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716e5b09-0f43-4eb4-a7bb-0b7975cfeef7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:37.810600 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716e5b09-0f43-4eb4-a7bb-0b7975cfeef7 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:37.831509 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75c7eab-5f87-4b60-9b8d-f8f57b3e1ebc admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:37.839107 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1160/2344] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:15:37 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 03:15:37.843972 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75c7eab-5f87-4b60-9b8d-f8f57b3e1ebc admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:37.853167 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da34e9ba-bf3c-4c24-b99d-a2e8ae5e3ef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:37.853950 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da34e9ba-bf3c-4c24-b99d-a2e8ae5e3ef2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:37.854236 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da34e9ba-bf3c-4c24-b99d-a2e8ae5e3ef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:37.854570 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da34e9ba-bf3c-4c24-b99d-a2e8ae5e3ef2 admin admin] PATH_INFO: `/v3/projects/c2fdf23f0560401ab8740dd8b024df6b/users/942747c1d4f94560be4afaeac55dd78c/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:37.873240 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da34e9ba-bf3c-4c24-b99d-a2e8ae5e3ef2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2fdf23f0560401ab8740dd8b024df6b, user_id=942747c1d4f94560be4afaeac55dd78c, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:37.876202 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da34e9ba-bf3c-4c24-b99d-a2e8ae5e3ef2 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:37.955536 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1185/2345] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:15:37 2022] PUT /identity/v3/projects/c2fdf23f0560401ab8740dd8b024df6b/users/942747c1d4f94560be4afaeac55dd78c/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:15:37.959677 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716e5b09-0f43-4eb4-a7bb-0b7975cfeef7 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:37.973120 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716e5b09-0f43-4eb4-a7bb-0b7975cfeef7 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:38.037905 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb29c75-24ce-4da9-bc30-0c3b577d5d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:38.037905 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb29c75-24ce-4da9-bc30-0c3b577d5d8a admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:38.037905 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb29c75-24ce-4da9-bc30-0c3b577d5d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:38.037905 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb29c75-24ce-4da9-bc30-0c3b577d5d8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:38.039203 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb29c75-24ce-4da9-bc30-0c3b577d5d8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:38.041016 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb29c75-24ce-4da9-bc30-0c3b577d5d8a admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:38.057792 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1161/2346] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:15:37 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:15:38.062181 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da34e9ba-bf3c-4c24-b99d-a2e8ae5e3ef2 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:38.072359 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb29c75-24ce-4da9-bc30-0c3b577d5d8a admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:38.076902 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da34e9ba-bf3c-4c24-b99d-a2e8ae5e3ef2 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:38.078172 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-5bb29c75-24ce-4da9-bc30-0c3b577d5d8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:38.086500 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797b48e5-1a66-4324-9bfa-178dacda3e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:38.088042 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797b48e5-1a66-4324-9bfa-178dacda3e45 admin admin] REQUEST_METHOD: `PUT` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:38.088573 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797b48e5-1a66-4324-9bfa-178dacda3e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:38.088687 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797b48e5-1a66-4324-9bfa-178dacda3e45 admin admin] PATH_INFO: `/v3/projects/c2fdf23f0560401ab8740dd8b024df6b/users/942747c1d4f94560be4afaeac55dd78c/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:38.090683 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797b48e5-1a66-4324-9bfa-178dacda3e45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2fdf23f0560401ab8740dd8b024df6b, user_id=942747c1d4f94560be4afaeac55dd78c, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:38.093324 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797b48e5-1a66-4324-9bfa-178dacda3e45 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:38.099226 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb29c75-24ce-4da9-bc30-0c3b577d5d8a admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:38.106423 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1186/2347] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:15:38 2022] PUT /identity/v3/projects/c2fdf23f0560401ab8740dd8b024df6b/users/942747c1d4f94560be4afaeac55dd78c/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:15:38.126096 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e60bf4c-79fe-460c-8056-59ad1b4fdc4c None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:38.126917 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e60bf4c-79fe-460c-8056-59ad1b4fdc4c None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:38.126917 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e60bf4c-79fe-460c-8056-59ad1b4fdc4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:38.145946 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:15:38.166167 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-5e60bf4c-79fe-460c-8056-59ad1b4fdc4c None None] MFA Rules not processed for user `942747c1d4f94560be4afaeac55dd78c`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:15:38.166370 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9667b5-2381-4241-ad96-2fa3d0ace97c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:38.167052 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9667b5-2381-4241-ad96-2fa3d0ace97c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:38.167366 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-5e60bf4c-79fe-460c-8056-59ad1b4fdc4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:38.167694 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9667b5-2381-4241-ad96-2fa3d0ace97c admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:38.168455 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9667b5-2381-4241-ad96-2fa3d0ace97c admin admin] PATH_INFO: `/v3/users/63a3c5f18b7b4cb0b411c68c1ef99afa` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:38.169372 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9667b5-2381-4241-ad96-2fa3d0ace97c admin admin] RBAC: Authorizing `identity:delete_user(user_id=63a3c5f18b7b4cb0b411c68c1ef99afa)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:38.171411 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9667b5-2381-4241-ad96-2fa3d0ace97c admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:38.269923 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1187/2348] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:15:38 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:15:38.303442 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e60bf4c-79fe-460c-8056-59ad1b4fdc4c None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:38.307592 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-5e60bf4c-79fe-460c-8056-59ad1b4fdc4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:38.386068 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:15:38.413836 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-2d9667b5-2381-4241-ad96-2fa3d0ace97c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63a3c5f18b7b4cb0b411c68c1ef99afa'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:38.427790 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e60bf4c-79fe-460c-8056-59ad1b4fdc4c None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:38.427790 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04140ec3-3fa5-440d-b49e-b76d204c06d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab599d98b0624a2e89090312bcab713f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:38.428520 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-2d9667b5-2381-4241-ad96-2fa3d0ace97c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63a3c5f18b7b4cb0b411c68c1ef99afa'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:38.428901 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04140ec3-3fa5-440d-b49e-b76d204c06d5 service cinder] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:38.428901 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04140ec3-3fa5-440d-b49e-b76d204c06d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:38.428901 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04140ec3-3fa5-440d-b49e-b76d204c06d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:38.439506 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-2d9667b5-2381-4241-ad96-2fa3d0ace97c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63a3c5f18b7b4cb0b411c68c1ef99afa'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:38.440538 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-2d9667b5-2381-4241-ad96-2fa3d0ace97c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63a3c5f18b7b4cb0b411c68c1ef99afa'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:38.493414 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04140ec3-3fa5-440d-b49e-b76d204c06d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:38.495284 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04140ec3-3fa5-440d-b49e-b76d204c06d5 service cinder] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:38.527588 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1188/2349] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:15:38 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:15:38.539541 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-2d9667b5-2381-4241-ad96-2fa3d0ace97c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63a3c5f18b7b4cb0b411c68c1ef99afa'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:38.547930 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1162/2350] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:15:38 2022] DELETE /identity/v3/users/63a3c5f18b7b4cb0b411c68c1ef99afa => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:15:38.552073 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3844bc11-739b-4b6b-878e-1e782d5faa4d None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:38.552471 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3844bc11-739b-4b6b-878e-1e782d5faa4d None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:38.552852 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3844bc11-739b-4b6b-878e-1e782d5faa4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:38.590743 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-3844bc11-739b-4b6b-878e-1e782d5faa4d None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:15:38.592112 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-3844bc11-739b-4b6b-878e-1e782d5faa4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:38.688617 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1189/2351] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:15:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:15:38.699864 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9667b5-2381-4241-ad96-2fa3d0ace97c admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:38.702783 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-2d9667b5-2381-4241-ad96-2fa3d0ace97c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:38.781444 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9667b5-2381-4241-ad96-2fa3d0ace97c admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:38.785445 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e96945-1b9c-43ad-8ae4-74f49da28cb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:38.786211 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e96945-1b9c-43ad-8ae4-74f49da28cb1 service neutron] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:38.786551 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e96945-1b9c-43ad-8ae4-74f49da28cb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:38.786831 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e96945-1b9c-43ad-8ae4-74f49da28cb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:38.796816 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e96945-1b9c-43ad-8ae4-74f49da28cb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:38.798389 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e96945-1b9c-43ad-8ae4-74f49da28cb1 service neutron] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:38.809143 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1163/2352] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:15:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:15:39.447463 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3844bc11-739b-4b6b-878e-1e782d5faa4d None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:39.450376 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-3844bc11-739b-4b6b-878e-1e782d5faa4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:39.485429 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3844bc11-739b-4b6b-878e-1e782d5faa4d None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:39.493470 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7025a4fc-35dc-4aca-8059-f9160ecbee81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:39.495215 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7025a4fc-35dc-4aca-8059-f9160ecbee81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:39.495687 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7025a4fc-35dc-4aca-8059-f9160ecbee81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:39.496168 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7025a4fc-35dc-4aca-8059-f9160ecbee81 admin admin] PATH_INFO: `/v3/projects/f5763416b30a4109a4859d989261e6ad` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:39.497624 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7025a4fc-35dc-4aca-8059-f9160ecbee81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5763416b30a4109a4859d989261e6ad)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:39.500266 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7025a4fc-35dc-4aca-8059-f9160ecbee81 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:39.716629 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-7025a4fc-35dc-4aca-8059-f9160ecbee81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5763416b30a4109a4859d989261e6ad', 'request_id': 'req-7025a4fc-35dc-4aca-8059-f9160ecbee81'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:39.728314 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-7025a4fc-35dc-4aca-8059-f9160ecbee81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5763416b30a4109a4859d989261e6ad', 'request_id': 'req-7025a4fc-35dc-4aca-8059-f9160ecbee81'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:39.827959 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-7025a4fc-35dc-4aca-8059-f9160ecbee81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5763416b30a4109a4859d989261e6ad', 'request_id': 'req-7025a4fc-35dc-4aca-8059-f9160ecbee81'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:39.852902 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-7025a4fc-35dc-4aca-8059-f9160ecbee81 admin admin] The token cache is being invalidate because project f5763416b30a4109a4859d989261e6ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81517) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:15:39.853202 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-7025a4fc-35dc-4aca-8059-f9160ecbee81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:39.854152 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-7025a4fc-35dc-4aca-8059-f9160ecbee81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:39.856792 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1190/2353] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:15:39 2022] DELETE /identity/v3/projects/f5763416b30a4109a4859d989261e6ad => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:15:39.860906 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e96945-1b9c-43ad-8ae4-74f49da28cb1 service neutron] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:39.876474 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e96945-1b9c-43ad-8ae4-74f49da28cb1 service neutron] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:39.958778 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f945488-8e88-4a40-87ed-483eb2470d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:39.959490 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f945488-8e88-4a40-87ed-483eb2470d9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:39.959818 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f945488-8e88-4a40-87ed-483eb2470d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:39.960058 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f945488-8e88-4a40-87ed-483eb2470d9d admin admin] PATH_INFO: `/v3/users/4b255ae026fb4c519ceb54fa393a5718` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:39.961453 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f945488-8e88-4a40-87ed-483eb2470d9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b255ae026fb4c519ceb54fa393a5718)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:39.963745 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f945488-8e88-4a40-87ed-483eb2470d9d admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:39.967040 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:15:40.228556 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-7f945488-8e88-4a40-87ed-483eb2470d9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b255ae026fb4c519ceb54fa393a5718'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:40.242879 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-7f945488-8e88-4a40-87ed-483eb2470d9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b255ae026fb4c519ceb54fa393a5718'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:40.266235 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-7f945488-8e88-4a40-87ed-483eb2470d9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b255ae026fb4c519ceb54fa393a5718'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:40.267809 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-7f945488-8e88-4a40-87ed-483eb2470d9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b255ae026fb4c519ceb54fa393a5718'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:40.369320 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-7f945488-8e88-4a40-87ed-483eb2470d9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b255ae026fb4c519ceb54fa393a5718'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:40.379892 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1164/2354] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:15:39 2022] DELETE /identity/v3/users/4b255ae026fb4c519ceb54fa393a5718 => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 03:15:40.437204 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7025a4fc-35dc-4aca-8059-f9160ecbee81 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:40.437204 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-7025a4fc-35dc-4aca-8059-f9160ecbee81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:40.457560 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7025a4fc-35dc-4aca-8059-f9160ecbee81 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:40.495142 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:15:40.609381 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3f1af9-3d96-4ee0-a216-3322e44eec38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:40.610059 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3f1af9-3d96-4ee0-a216-3322e44eec38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:40.610515 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3f1af9-3d96-4ee0-a216-3322e44eec38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:40.610935 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3f1af9-3d96-4ee0-a216-3322e44eec38 admin admin] PATH_INFO: `/v3/users/927637d7c6274d3c821814c5cb7dbea3` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:40.613839 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3f1af9-3d96-4ee0-a216-3322e44eec38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=927637d7c6274d3c821814c5cb7dbea3)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:40.616013 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3f1af9-3d96-4ee0-a216-3322e44eec38 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:40.850626 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f945488-8e88-4a40-87ed-483eb2470d9d admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:40.852844 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-7f945488-8e88-4a40-87ed-483eb2470d9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:40.960570 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f945488-8e88-4a40-87ed-483eb2470d9d admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:40.964574 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d87733d-cf53-4388-bd64-74be0a615703 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab599d98b0624a2e89090312bcab713f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:40.965337 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d87733d-cf53-4388-bd64-74be0a615703 service cinder] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:40.965905 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d87733d-cf53-4388-bd64-74be0a615703 service cinder] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:40.966287 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d87733d-cf53-4388-bd64-74be0a615703 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:40.968351 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-9d87733d-cf53-4388-bd64-74be0a615703 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:41.029658 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-df3f1af9-3d96-4ee0-a216-3322e44eec38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '927637d7c6274d3c821814c5cb7dbea3'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:41.034537 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-df3f1af9-3d96-4ee0-a216-3322e44eec38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '927637d7c6274d3c821814c5cb7dbea3'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:41.106816 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d87733d-cf53-4388-bd64-74be0a615703 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:41.108327 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d87733d-cf53-4388-bd64-74be0a615703 service cinder] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:41.120782 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1165/2355] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:15:40 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 03:15:41.121778 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d87733d-cf53-4388-bd64-74be0a615703 service cinder] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:41.126512 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-9d87733d-cf53-4388-bd64-74be0a615703 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:41.137639 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d87733d-cf53-4388-bd64-74be0a615703 service cinder] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:41.142252 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1af832-88af-4414-9576-02ae6f400b72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:41.142909 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1af832-88af-4414-9576-02ae6f400b72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:41.143845 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1af832-88af-4414-9576-02ae6f400b72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:41.144126 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1af832-88af-4414-9576-02ae6f400b72 admin admin] PATH_INFO: `/v3/projects/f04b00e4841d4f68a5dede354be864ff` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:41.144604 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-df3f1af9-3d96-4ee0-a216-3322e44eec38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '927637d7c6274d3c821814c5cb7dbea3'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:41.145258 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1af832-88af-4414-9576-02ae6f400b72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f04b00e4841d4f68a5dede354be864ff)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:41.145420 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-df3f1af9-3d96-4ee0-a216-3322e44eec38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '927637d7c6274d3c821814c5cb7dbea3'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:41.147201 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1af832-88af-4414-9576-02ae6f400b72 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:41.153607 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-df3f1af9-3d96-4ee0-a216-3322e44eec38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '927637d7c6274d3c821814c5cb7dbea3'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:41.173298 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1191/2356] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:15:40 2022] DELETE /identity/v3/users/927637d7c6274d3c821814c5cb7dbea3 => generated 0 bytes in 741 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:15:41.177740 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9948ee63-5581-4446-8f37-27fb92cf3916 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:41.178165 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9948ee63-5581-4446-8f37-27fb92cf3916 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:41.179048 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9948ee63-5581-4446-8f37-27fb92cf3916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:41.201678 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-9948ee63-5581-4446-8f37-27fb92cf3916 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:15:41.203104 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-9948ee63-5581-4446-8f37-27fb92cf3916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:41.248533 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:15:41.266054 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-4c1af832-88af-4414-9576-02ae6f400b72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f04b00e4841d4f68a5dede354be864ff', 'request_id': 'req-4c1af832-88af-4414-9576-02ae6f400b72'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:41.274815 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-4c1af832-88af-4414-9576-02ae6f400b72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f04b00e4841d4f68a5dede354be864ff', 'request_id': 'req-4c1af832-88af-4414-9576-02ae6f400b72'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:41.325323 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1192/2357] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:15:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 03:15:41.340080 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9948ee63-5581-4446-8f37-27fb92cf3916 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:41.344694 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-9948ee63-5581-4446-8f37-27fb92cf3916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:41.353973 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-4c1af832-88af-4414-9576-02ae6f400b72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f04b00e4841d4f68a5dede354be864ff', 'request_id': 'req-4c1af832-88af-4414-9576-02ae6f400b72'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:41.393675 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-4c1af832-88af-4414-9576-02ae6f400b72 admin admin] The token cache is being invalidate because project f04b00e4841d4f68a5dede354be864ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:15:41.394650 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-4c1af832-88af-4414-9576-02ae6f400b72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:41.395524 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-4c1af832-88af-4414-9576-02ae6f400b72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:41.397238 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1166/2358] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:15:41 2022] DELETE /identity/v3/projects/f04b00e4841d4f68a5dede354be864ff => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:15:41.403395 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d889b1-77ad-4489-b454-5a43da44fb88 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:41.405023 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d889b1-77ad-4489-b454-5a43da44fb88 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:41.406105 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d889b1-77ad-4489-b454-5a43da44fb88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:41.446977 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9948ee63-5581-4446-8f37-27fb92cf3916 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:41.453416 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-92d889b1-77ad-4489-b454-5a43da44fb88 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:15:41.458018 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-92d889b1-77ad-4489-b454-5a43da44fb88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:41.535967 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f157e5-142d-44c4-88dd-da46e3b29afb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:41.537408 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f157e5-142d-44c4-88dd-da46e3b29afb service neutron] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:41.537957 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f157e5-142d-44c4-88dd-da46e3b29afb service neutron] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:41.538896 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f157e5-142d-44c4-88dd-da46e3b29afb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:41.541493 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-49f157e5-142d-44c4-88dd-da46e3b29afb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:41.571258 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1167/2359] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:15:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 03:15:41.573931 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71440f92-c00f-4dc8-9384-563c9bf3b30f None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:41.574221 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71440f92-c00f-4dc8-9384-563c9bf3b30f None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:41.575107 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71440f92-c00f-4dc8-9384-563c9bf3b30f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:41.618283 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-71440f92-c00f-4dc8-9384-563c9bf3b30f None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:15:41.622432 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-71440f92-c00f-4dc8-9384-563c9bf3b30f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:41.635261 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:15:41.641491 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1168/2360] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:15:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:15:41.647641 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71440f92-c00f-4dc8-9384-563c9bf3b30f None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:41.674976 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f157e5-142d-44c4-88dd-da46e3b29afb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:41.676770 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f157e5-142d-44c4-88dd-da46e3b29afb service neutron] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:41.680607 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71440f92-c00f-4dc8-9384-563c9bf3b30f None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:41.692767 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1193/2361] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:15:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:15:41.695743 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e931cb1e-cc0c-44e6-9747-78e077c38917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:41.696513 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e931cb1e-cc0c-44e6-9747-78e077c38917 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:41.696798 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e931cb1e-cc0c-44e6-9747-78e077c38917 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:41.697380 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e931cb1e-cc0c-44e6-9747-78e077c38917 admin admin] PATH_INFO: `/v3/domains` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:41.698655 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e931cb1e-cc0c-44e6-9747-78e077c38917 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 03:15:41.698978 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e931cb1e-cc0c-44e6-9747-78e077c38917 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:41.700685 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e931cb1e-cc0c-44e6-9747-78e077c38917 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:41.717885 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1169/2362] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Oct 14 03:15:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 03:15:41.722908 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f157e5-142d-44c4-88dd-da46e3b29afb service neutron] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:41.734342 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee950b-e3e3-411c-be52-d42037210886 None None] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:41.734588 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee950b-e3e3-411c-be52-d42037210886 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:41.734859 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee950b-e3e3-411c-be52-d42037210886 None None] PATH_INFO: `/` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:41.736604 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f157e5-142d-44c4-88dd-da46e3b29afb service neutron] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:41.740855 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1170/2363] 10.222.0.39 () {54 vars in 961 bytes} [Fri Oct 14 03:15:41 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 14 03:15:41.743831 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7a93b1-a0a4-4bde-a03f-b7c355887f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:41.744547 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7a93b1-a0a4-4bde-a03f-b7c355887f95 admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:41.744833 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7a93b1-a0a4-4bde-a03f-b7c355887f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:41.745162 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7a93b1-a0a4-4bde-a03f-b7c355887f95 admin admin] PATH_INFO: `/v3/projects` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:41.746002 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7a93b1-a0a4-4bde-a03f-b7c355887f95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:41.749650 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7a93b1-a0a4-4bde-a03f-b7c355887f95 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:41.750572 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c210b-cd1d-467e-b21a-261923b8540d None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:41.750866 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c210b-cd1d-467e-b21a-261923b8540d None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:41.751145 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c210b-cd1d-467e-b21a-261923b8540d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:41.865422 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1194/2364] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:15:41 2022] POST /identity/v3/projects => generated 373 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:15:41.869719 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7a93b1-a0a4-4bde-a03f-b7c355887f95 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:41.877463 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-465c210b-cd1d-467e-b21a-261923b8540d None None] MFA Rules not processed for user `922ef7811aed450baff7f936011b0af1`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:15:41.881111 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-465c210b-cd1d-467e-b21a-261923b8540d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:41.991349 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7a93b1-a0a4-4bde-a03f-b7c355887f95 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:41.998833 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241379d9-3623-4e6c-ab6d-e7fd472dbb1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:42.000705 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241379d9-3623-4e6c-ab6d-e7fd472dbb1b admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:42.001198 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241379d9-3623-4e6c-ab6d-e7fd472dbb1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:42.001668 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241379d9-3623-4e6c-ab6d-e7fd472dbb1b admin admin] PATH_INFO: `/v3/users` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:42.008715 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241379d9-3623-4e6c-ab6d-e7fd472dbb1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:42.012393 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241379d9-3623-4e6c-ab6d-e7fd472dbb1b admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:42.068623 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1171/2365] 10.222.0.39 () {58 vars in 1058 bytes} [Fri Oct 14 03:15:41 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:15:42.078752 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465c210b-cd1d-467e-b21a-261923b8540d None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:42.098474 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1195/2366] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:15:41 2022] POST /identity/v3/users => generated 287 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 03:15:42.102172 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241379d9-3623-4e6c-ab6d-e7fd472dbb1b admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:42.103622 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465c210b-cd1d-467e-b21a-261923b8540d None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:42.112812 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fdd2f2-7376-4391-9369-3f7c8c9fc5b9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '922ef7811aed450baff7f936011b0af1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:42.113431 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fdd2f2-7376-4391-9369-3f7c8c9fc5b9 service placement] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:42.113731 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fdd2f2-7376-4391-9369-3f7c8c9fc5b9 service placement] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:42.114103 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fdd2f2-7376-4391-9369-3f7c8c9fc5b9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:42.120720 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241379d9-3623-4e6c-ab6d-e7fd472dbb1b admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:42.122169 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-18fdd2f2-7376-4391-9369-3f7c8c9fc5b9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:42.150540 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e59396-85c2-4613-be02-f9549c9218da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:42.155242 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e59396-85c2-4613-be02-f9549c9218da admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:42.157828 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e59396-85c2-4613-be02-f9549c9218da admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:42.158671 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e59396-85c2-4613-be02-f9549c9218da admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:42.159543 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e59396-85c2-4613-be02-f9549c9218da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:42.160154 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fdd2f2-7376-4391-9369-3f7c8c9fc5b9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:42.161815 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fdd2f2-7376-4391-9369-3f7c8c9fc5b9 service placement] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:42.164685 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e59396-85c2-4613-be02-f9549c9218da admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:42.183976 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1172/2367] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:15:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 03:15:42.195236 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1196/2368] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:15:42 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 03:15:42.200790 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fdd2f2-7376-4391-9369-3f7c8c9fc5b9 service placement] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:42.233602 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fdd2f2-7376-4391-9369-3f7c8c9fc5b9 service placement] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:42.247608 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73db3eaa-9f1e-44be-9d2b-70c234132bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:42.248739 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db3eaa-9f1e-44be-9d2b-70c234132bbf admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:42.249129 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db3eaa-9f1e-44be-9d2b-70c234132bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:42.249705 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db3eaa-9f1e-44be-9d2b-70c234132bbf admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:42.250504 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73db3eaa-9f1e-44be-9d2b-70c234132bbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:42.253002 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73db3eaa-9f1e-44be-9d2b-70c234132bbf admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:42.303325 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1173/2369] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:15:42 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 03:15:42.310924 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e59396-85c2-4613-be02-f9549c9218da admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:42.318019 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e59396-85c2-4613-be02-f9549c9218da admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:42.337717 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ae204e-7bd0-4454-b82f-f1d718465eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:42.338677 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ae204e-7bd0-4454-b82f-f1d718465eff admin admin] REQUEST_METHOD: `PUT` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:42.339198 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ae204e-7bd0-4454-b82f-f1d718465eff admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:42.339486 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ae204e-7bd0-4454-b82f-f1d718465eff admin admin] PATH_INFO: `/v3/projects/f7f11955b8c9406daa11fe9576edef04/users/440a43fbd0aa4645895da5057bd41870/roles/9bd0470b544240209a420f640c79c06c` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:42.375662 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ae204e-7bd0-4454-b82f-f1d718465eff admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7f11955b8c9406daa11fe9576edef04, user_id=440a43fbd0aa4645895da5057bd41870, role_id=9bd0470b544240209a420f640c79c06c)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:42.378613 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ae204e-7bd0-4454-b82f-f1d718465eff admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:42.402653 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73db3eaa-9f1e-44be-9d2b-70c234132bbf admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:42.404482 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-73db3eaa-9f1e-44be-9d2b-70c234132bbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:42.447579 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73db3eaa-9f1e-44be-9d2b-70c234132bbf admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:42.459866 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e811239-7cfc-433d-97ed-ae7ac659b86c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:42.468175 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e811239-7cfc-433d-97ed-ae7ac659b86c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:42.468550 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e811239-7cfc-433d-97ed-ae7ac659b86c admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:42.468836 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e811239-7cfc-433d-97ed-ae7ac659b86c admin admin] PATH_INFO: `/v3/projects/68d5bc3ff5d44d5f8ee2821597888c08` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:42.470752 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e811239-7cfc-433d-97ed-ae7ac659b86c admin admin] RBAC: Authorizing `identity:delete_project(project_id=68d5bc3ff5d44d5f8ee2821597888c08)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:42.472537 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1197/2370] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:15:42 2022] PUT /identity/v3/projects/f7f11955b8c9406daa11fe9576edef04/users/440a43fbd0aa4645895da5057bd41870/roles/9bd0470b544240209a420f640c79c06c => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:15:42.474489 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e811239-7cfc-433d-97ed-ae7ac659b86c admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:42.478100 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bf57fd-7467-4a8f-afd4-161dff7d0afc None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:42.478677 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bf57fd-7467-4a8f-afd4-161dff7d0afc None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:42.479176 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bf57fd-7467-4a8f-afd4-161dff7d0afc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:42.512959 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-87bf57fd-7467-4a8f-afd4-161dff7d0afc None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:15:42.514539 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-87bf57fd-7467-4a8f-afd4-161dff7d0afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:42.608511 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1198/2371] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:15:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:15:42.620292 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bf57fd-7467-4a8f-afd4-161dff7d0afc None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:42.621681 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-87bf57fd-7467-4a8f-afd4-161dff7d0afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:42.686636 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bf57fd-7467-4a8f-afd4-161dff7d0afc None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:42.690789 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652c6c1c-215d-43e2-92bc-d611edef83fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:42.691409 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652c6c1c-215d-43e2-92bc-d611edef83fc service neutron] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:42.691696 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652c6c1c-215d-43e2-92bc-d611edef83fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:42.692041 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652c6c1c-215d-43e2-92bc-d611edef83fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:42.704115 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652c6c1c-215d-43e2-92bc-d611edef83fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:42.706268 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652c6c1c-215d-43e2-92bc-d611edef83fc service neutron] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:42.719727 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1199/2372] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:15:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 03:15:42.722647 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73e803f-a370-401f-ac91-b1c16a53538c None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:42.723327 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73e803f-a370-401f-ac91-b1c16a53538c None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:42.723770 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73e803f-a370-401f-ac91-b1c16a53538c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:42.764063 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-f73e803f-a370-401f-ac91-b1c16a53538c None None] MFA Rules not processed for user `4b0a436168bb45aa9635df4774e275dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:15:42.765372 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-f73e803f-a370-401f-ac91-b1c16a53538c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:42.822403 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-4e811239-7cfc-433d-97ed-ae7ac659b86c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68d5bc3ff5d44d5f8ee2821597888c08', 'request_id': 'req-4e811239-7cfc-433d-97ed-ae7ac659b86c'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:42.830236 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-4e811239-7cfc-433d-97ed-ae7ac659b86c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68d5bc3ff5d44d5f8ee2821597888c08', 'request_id': 'req-4e811239-7cfc-433d-97ed-ae7ac659b86c'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:42.840367 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:15:42.849452 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1200/2373] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:15:42 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:15:42.866011 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73e803f-a370-401f-ac91-b1c16a53538c None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:42.867740 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-f73e803f-a370-401f-ac91-b1c16a53538c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:42.901917 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-4e811239-7cfc-433d-97ed-ae7ac659b86c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68d5bc3ff5d44d5f8ee2821597888c08', 'request_id': 'req-4e811239-7cfc-433d-97ed-ae7ac659b86c'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:42.914729 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-4e811239-7cfc-433d-97ed-ae7ac659b86c admin admin] The token cache is being invalidate because project 68d5bc3ff5d44d5f8ee2821597888c08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:15:42.915068 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-4e811239-7cfc-433d-97ed-ae7ac659b86c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:42.915822 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-4e811239-7cfc-433d-97ed-ae7ac659b86c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:42.920645 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1174/2374] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:15:42 2022] DELETE /identity/v3/projects/68d5bc3ff5d44d5f8ee2821597888c08 => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:15:42.920645 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e811239-7cfc-433d-97ed-ae7ac659b86c admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:42.922065 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-4e811239-7cfc-433d-97ed-ae7ac659b86c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:43.011798 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e811239-7cfc-433d-97ed-ae7ac659b86c admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:43.015630 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e585b6-5789-40ef-9411-a8ef8f2c392d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab599d98b0624a2e89090312bcab713f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:43.016407 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e585b6-5789-40ef-9411-a8ef8f2c392d service cinder] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:43.016658 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e585b6-5789-40ef-9411-a8ef8f2c392d service cinder] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:43.016929 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e585b6-5789-40ef-9411-a8ef8f2c392d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:43.019084 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-e9e585b6-5789-40ef-9411-a8ef8f2c392d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:43.041043 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73e803f-a370-401f-ac91-b1c16a53538c None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:43.092128 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e585b6-5789-40ef-9411-a8ef8f2c392d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:43.094452 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e585b6-5789-40ef-9411-a8ef8f2c392d service cinder] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:43.111818 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1175/2375] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:15:42 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:15:43.114213 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1789ae-f59c-448a-97e9-89e1c1ceb753 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:43.114735 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1789ae-f59c-448a-97e9-89e1c1ceb753 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:43.115131 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1789ae-f59c-448a-97e9-89e1c1ceb753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:43.140444 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-fc1789ae-f59c-448a-97e9-89e1c1ceb753 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:15:43.142081 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-fc1789ae-f59c-448a-97e9-89e1c1ceb753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:43.154135 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e84662d-6e92-4d6f-829e-7767defbe077 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b07bf0faca47b28b50990d1720853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:43.155011 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e84662d-6e92-4d6f-829e-7767defbe077 service nova] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:43.155369 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e84662d-6e92-4d6f-829e-7767defbe077 service nova] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:43.155716 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e84662d-6e92-4d6f-829e-7767defbe077 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:43.157888 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-2e84662d-6e92-4d6f-829e-7767defbe077 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:43.191440 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:15:43.217170 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1176/2376] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:15:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:15:43.218080 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7c778a-9bfc-4036-9378-7e29619fa40f None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:43.218366 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7c778a-9bfc-4036-9378-7e29619fa40f None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:43.218639 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7c778a-9bfc-4036-9378-7e29619fa40f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:43.240863 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-2f7c778a-9bfc-4036-9378-7e29619fa40f None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:15:43.241132 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e84662d-6e92-4d6f-829e-7767defbe077 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:43.242086 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-2f7c778a-9bfc-4036-9378-7e29619fa40f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:43.242853 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e84662d-6e92-4d6f-829e-7767defbe077 service nova] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:43.252331 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1177/2377] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:15:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 03:15:43.254652 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1201/2378] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:15:42 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 03:15:43.256469 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7c778a-9bfc-4036-9378-7e29619fa40f None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:43.265999 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7c778a-9bfc-4036-9378-7e29619fa40f None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:43.272220 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8225545a-b257-4c30-b00f-86442a12c1c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:43.272739 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8225545a-b257-4c30-b00f-86442a12c1c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:43.273070 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8225545a-b257-4c30-b00f-86442a12c1c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:43.273365 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8225545a-b257-4c30-b00f-86442a12c1c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:43.274073 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8225545a-b257-4c30-b00f-86442a12c1c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 03:15:43.274379 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8225545a-b257-4c30-b00f-86442a12c1c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:43.276013 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8225545a-b257-4c30-b00f-86442a12c1c8 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:43.292586 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1178/2379] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Oct 14 03:15:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 14 03:15:43.298174 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e84662d-6e92-4d6f-829e-7767defbe077 service nova] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:43.310364 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e84662d-6e92-4d6f-829e-7767defbe077 service nova] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:43.315086 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e60ca6-1b7d-4fb2-b13a-7c2773cd1202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:43.315633 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e60ca6-1b7d-4fb2-b13a-7c2773cd1202 admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:43.315919 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e60ca6-1b7d-4fb2-b13a-7c2773cd1202 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:43.316197 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e60ca6-1b7d-4fb2-b13a-7c2773cd1202 admin admin] PATH_INFO: `/v3/projects` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:43.316846 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e60ca6-1b7d-4fb2-b13a-7c2773cd1202 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:43.318797 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e60ca6-1b7d-4fb2-b13a-7c2773cd1202 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:43.410716 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1202/2380] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:15:43 2022] POST /identity/v3/projects => generated 361 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 03:15:43.414945 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8225545a-b257-4c30-b00f-86442a12c1c8 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:43.425270 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8225545a-b257-4c30-b00f-86442a12c1c8 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:43.492387 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09e6f47-8f4b-4293-ba29-698668c11ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:43.492387 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09e6f47-8f4b-4293-ba29-698668c11ef0 admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:43.492387 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09e6f47-8f4b-4293-ba29-698668c11ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:43.492387 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09e6f47-8f4b-4293-ba29-698668c11ef0 admin admin] PATH_INFO: `/v3/users` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:43.492850 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09e6f47-8f4b-4293-ba29-698668c11ef0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:43.494415 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09e6f47-8f4b-4293-ba29-698668c11ef0 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:43.609526 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1179/2381] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:15:43 2022] POST /identity/v3/users => generated 281 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:15:43.615348 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e60ca6-1b7d-4fb2-b13a-7c2773cd1202 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:43.628852 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e60ca6-1b7d-4fb2-b13a-7c2773cd1202 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:43.634215 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2656cddd-21e2-4ee2-85e8-cafe494aa29e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:43.634933 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2656cddd-21e2-4ee2-85e8-cafe494aa29e admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:43.635291 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2656cddd-21e2-4ee2-85e8-cafe494aa29e admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:43.635693 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2656cddd-21e2-4ee2-85e8-cafe494aa29e admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:43.636402 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2656cddd-21e2-4ee2-85e8-cafe494aa29e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:43.638188 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2656cddd-21e2-4ee2-85e8-cafe494aa29e admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:43.650719 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1203/2382] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:15:43 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:15:43.655508 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09e6f47-8f4b-4293-ba29-698668c11ef0 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:43.667587 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09e6f47-8f4b-4293-ba29-698668c11ef0 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:43.672852 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c39f295-619a-4160-8390-5c5d8d0c44cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:43.673640 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c39f295-619a-4160-8390-5c5d8d0c44cd admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:43.674056 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c39f295-619a-4160-8390-5c5d8d0c44cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:43.674261 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c39f295-619a-4160-8390-5c5d8d0c44cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:43.675017 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c39f295-619a-4160-8390-5c5d8d0c44cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:43.676672 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c39f295-619a-4160-8390-5c5d8d0c44cd admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:43.695309 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1180/2383] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:15:43 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:15:43.699681 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2656cddd-21e2-4ee2-85e8-cafe494aa29e admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:43.711501 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2656cddd-21e2-4ee2-85e8-cafe494aa29e admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:43.716417 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b911a1e7-428a-4182-9ab7-41553605c99a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:43.717140 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b911a1e7-428a-4182-9ab7-41553605c99a admin admin] REQUEST_METHOD: `PUT` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:43.717485 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b911a1e7-428a-4182-9ab7-41553605c99a admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:43.717850 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b911a1e7-428a-4182-9ab7-41553605c99a admin admin] PATH_INFO: `/v3/projects/0554790d9e0b4135a87c7e96fe823835/users/755d23f138ea4a74a414548b642bb6f1/roles/9bd0470b544240209a420f640c79c06c` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:43.735160 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b911a1e7-428a-4182-9ab7-41553605c99a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0554790d9e0b4135a87c7e96fe823835, user_id=755d23f138ea4a74a414548b642bb6f1, role_id=9bd0470b544240209a420f640c79c06c)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:43.738622 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b911a1e7-428a-4182-9ab7-41553605c99a admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:43.881874 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1204/2384] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:15:43 2022] PUT /identity/v3/projects/0554790d9e0b4135a87c7e96fe823835/users/755d23f138ea4a74a414548b642bb6f1/roles/9bd0470b544240209a420f640c79c06c => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 03:15:43.893906 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f33cb32-e16d-46d8-b31a-a288ab7948fa None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:43.900605 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f33cb32-e16d-46d8-b31a-a288ab7948fa None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:43.900605 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f33cb32-e16d-46d8-b31a-a288ab7948fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:43.928410 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-2f33cb32-e16d-46d8-b31a-a288ab7948fa None None] MFA Rules not processed for user `755d23f138ea4a74a414548b642bb6f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:15:43.929839 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-2f33cb32-e16d-46d8-b31a-a288ab7948fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:44.027191 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1181/2385] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:15:43 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:15:44.062172 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b911a1e7-428a-4182-9ab7-41553605c99a admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:44.130924 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b911a1e7-428a-4182-9ab7-41553605c99a admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:44.134590 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6584c7-261a-45d1-b19e-3399219d343e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab599d98b0624a2e89090312bcab713f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:44.135191 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6584c7-261a-45d1-b19e-3399219d343e service cinder] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:44.135469 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6584c7-261a-45d1-b19e-3399219d343e service cinder] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:44.135881 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6584c7-261a-45d1-b19e-3399219d343e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:44.145411 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6584c7-261a-45d1-b19e-3399219d343e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:44.147088 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6584c7-261a-45d1-b19e-3399219d343e service cinder] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:44.157449 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1205/2386] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:15:44 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:15:44.173962 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f33cb32-e16d-46d8-b31a-a288ab7948fa None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:44.239610 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f33cb32-e16d-46d8-b31a-a288ab7948fa None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:44.244129 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dcbf3a-cd30-43a8-8557-058ec7e6ff71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:44.244750 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dcbf3a-cd30-43a8-8557-058ec7e6ff71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:44.245029 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dcbf3a-cd30-43a8-8557-058ec7e6ff71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:44.245299 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dcbf3a-cd30-43a8-8557-058ec7e6ff71 admin admin] PATH_INFO: `/v3/users/755d23f138ea4a74a414548b642bb6f1` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:44.246543 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dcbf3a-cd30-43a8-8557-058ec7e6ff71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=755d23f138ea4a74a414548b642bb6f1)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:44.248440 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dcbf3a-cd30-43a8-8557-058ec7e6ff71 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:44.387645 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ba253a-2565-42b6-b2d6-b125b32405c2 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:44.387930 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ba253a-2565-42b6-b2d6-b125b32405c2 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:44.388226 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ba253a-2565-42b6-b2d6-b125b32405c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:44.420227 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-d9ba253a-2565-42b6-b2d6-b125b32405c2 None None] MFA Rules not processed for user `440a43fbd0aa4645895da5057bd41870`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:15:44.421543 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-d9ba253a-2565-42b6-b2d6-b125b32405c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:44.488642 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-e9dcbf3a-cd30-43a8-8557-058ec7e6ff71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '755d23f138ea4a74a414548b642bb6f1'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:44.497234 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-e9dcbf3a-cd30-43a8-8557-058ec7e6ff71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '755d23f138ea4a74a414548b642bb6f1'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:44.511621 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-e9dcbf3a-cd30-43a8-8557-058ec7e6ff71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '755d23f138ea4a74a414548b642bb6f1'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:44.512544 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-e9dcbf3a-cd30-43a8-8557-058ec7e6ff71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '755d23f138ea4a74a414548b642bb6f1'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:44.529697 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1206/2387] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:15:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 03:15:44.534862 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ba253a-2565-42b6-b2d6-b125b32405c2 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:44.536229 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-d9ba253a-2565-42b6-b2d6-b125b32405c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:44.550441 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ba253a-2565-42b6-b2d6-b125b32405c2 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:44.555382 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61795a57-0424-4300-bc14-a115992f6a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:44.556066 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61795a57-0424-4300-bc14-a115992f6a85 admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:44.556347 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61795a57-0424-4300-bc14-a115992f6a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:44.556622 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61795a57-0424-4300-bc14-a115992f6a85 admin admin] PATH_INFO: `/v3/projects` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:44.557367 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61795a57-0424-4300-bc14-a115992f6a85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:44.564488 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61795a57-0424-4300-bc14-a115992f6a85 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:44.606244 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-e9dcbf3a-cd30-43a8-8557-058ec7e6ff71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '755d23f138ea4a74a414548b642bb6f1'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:44.613446 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1182/2388] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:15:44 2022] DELETE /identity/v3/users/755d23f138ea4a74a414548b642bb6f1 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:15:44.618142 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a0a515-5e6e-4149-aaaa-53a9e41105e2 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:44.618427 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a0a515-5e6e-4149-aaaa-53a9e41105e2 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:44.618702 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a0a515-5e6e-4149-aaaa-53a9e41105e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:44.640479 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-01a0a515-5e6e-4149-aaaa-53a9e41105e2 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:15:44.641702 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-01a0a515-5e6e-4149-aaaa-53a9e41105e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:44.667638 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1207/2389] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:15:44 2022] POST /identity/v3/projects => generated 371 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 03:15:44.671717 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61795a57-0424-4300-bc14-a115992f6a85 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:44.739436 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1183/2390] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:15:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 03:15:44.745177 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61795a57-0424-4300-bc14-a115992f6a85 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:44.749569 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a0a515-5e6e-4149-aaaa-53a9e41105e2 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:44.760720 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1624af8c-362d-4389-9ab9-b3ed4cf211fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:44.760932 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-01a0a515-5e6e-4149-aaaa-53a9e41105e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:44.761037 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1624af8c-362d-4389-9ab9-b3ed4cf211fc admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:44.761037 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1624af8c-362d-4389-9ab9-b3ed4cf211fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:44.761037 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1624af8c-362d-4389-9ab9-b3ed4cf211fc admin admin] PATH_INFO: `/v3/users` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:44.761037 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1624af8c-362d-4389-9ab9-b3ed4cf211fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:44.761037 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1624af8c-362d-4389-9ab9-b3ed4cf211fc admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:44.781952 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:15:44.820786 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a0a515-5e6e-4149-aaaa-53a9e41105e2 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:44.872138 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e16892-d6a5-4ccb-90c5-e3694b6af86e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:44.872759 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e16892-d6a5-4ccb-90c5-e3694b6af86e service neutron] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:44.873042 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e16892-d6a5-4ccb-90c5-e3694b6af86e service neutron] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:44.873318 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e16892-d6a5-4ccb-90c5-e3694b6af86e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:44.883102 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e16892-d6a5-4ccb-90c5-e3694b6af86e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:44.884577 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e16892-d6a5-4ccb-90c5-e3694b6af86e service neutron] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:44.894822 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1184/2391] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:15:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:15:44.928487 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1208/2392] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:15:44 2022] POST /identity/v3/users => generated 292 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 03:15:44.930972 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e16892-d6a5-4ccb-90c5-e3694b6af86e service neutron] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:44.941952 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e16892-d6a5-4ccb-90c5-e3694b6af86e service neutron] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:44.946979 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1975b649-c77d-45f4-913d-8ed6b4e519b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:44.947663 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1975b649-c77d-45f4-913d-8ed6b4e519b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:44.948046 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1975b649-c77d-45f4-913d-8ed6b4e519b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:44.948390 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1975b649-c77d-45f4-913d-8ed6b4e519b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:44.949123 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1975b649-c77d-45f4-913d-8ed6b4e519b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:44.950863 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1975b649-c77d-45f4-913d-8ed6b4e519b7 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:44.962203 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1185/2393] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:15:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:15:44.967205 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1624af8c-362d-4389-9ab9-b3ed4cf211fc admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:44.978245 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1624af8c-362d-4389-9ab9-b3ed4cf211fc admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:44.983815 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf46929-3bb5-468d-b337-758fbfb86f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:44.984750 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf46929-3bb5-468d-b337-758fbfb86f60 admin admin] REQUEST_METHOD: `PUT` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:44.986960 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf46929-3bb5-468d-b337-758fbfb86f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:44.986960 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf46929-3bb5-468d-b337-758fbfb86f60 admin admin] PATH_INFO: `/v3/projects/ec85ee91ec734d0dbd86da9a12f44545/users/41f5223c01ba410abe3539cccb0e9a25/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:45.006183 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf46929-3bb5-468d-b337-758fbfb86f60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec85ee91ec734d0dbd86da9a12f44545, user_id=41f5223c01ba410abe3539cccb0e9a25, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:45.007534 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf46929-3bb5-468d-b337-758fbfb86f60 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:45.125698 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:15:45.134247 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1209/2394] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:15:44 2022] PUT /identity/v3/projects/ec85ee91ec734d0dbd86da9a12f44545/users/41f5223c01ba410abe3539cccb0e9a25/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:15:45.135733 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1975b649-c77d-45f4-913d-8ed6b4e519b7 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:45.147319 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1975b649-c77d-45f4-913d-8ed6b4e519b7 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:45.220485 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8977484e-494c-478b-b400-c7961a03a61b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:45.222300 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8977484e-494c-478b-b400-c7961a03a61b admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:45.222615 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8977484e-494c-478b-b400-c7961a03a61b admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:45.222863 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8977484e-494c-478b-b400-c7961a03a61b admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:45.223566 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8977484e-494c-478b-b400-c7961a03a61b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:45.225172 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8977484e-494c-478b-b400-c7961a03a61b admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:45.242004 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1186/2395] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:15:45 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:15:45.246619 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf46929-3bb5-468d-b337-758fbfb86f60 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:45.258876 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf46929-3bb5-468d-b337-758fbfb86f60 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:45.271308 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab98cfe3-acc7-4837-ba87-dff528e126ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:45.272515 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab98cfe3-acc7-4837-ba87-dff528e126ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:45.272915 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab98cfe3-acc7-4837-ba87-dff528e126ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:45.273927 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab98cfe3-acc7-4837-ba87-dff528e126ec admin admin] PATH_INFO: `/v3/projects/ec85ee91ec734d0dbd86da9a12f44545/users/41f5223c01ba410abe3539cccb0e9a25/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:45.276217 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab98cfe3-acc7-4837-ba87-dff528e126ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec85ee91ec734d0dbd86da9a12f44545, user_id=41f5223c01ba410abe3539cccb0e9a25, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:45.280093 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab98cfe3-acc7-4837-ba87-dff528e126ec admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:45.293216 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1210/2396] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:15:45 2022] PUT /identity/v3/projects/ec85ee91ec734d0dbd86da9a12f44545/users/41f5223c01ba410abe3539cccb0e9a25/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:15:45.525320 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8977484e-494c-478b-b400-c7961a03a61b admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:45.527271 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-8977484e-494c-478b-b400-c7961a03a61b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:45.544501 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8977484e-494c-478b-b400-c7961a03a61b admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:45.612681 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3bd240-657e-48cb-8fb7-0ed262878a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:45.613448 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3bd240-657e-48cb-8fb7-0ed262878a09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:45.613857 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3bd240-657e-48cb-8fb7-0ed262878a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:45.614247 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3bd240-657e-48cb-8fb7-0ed262878a09 admin admin] PATH_INFO: `/v3/projects/0554790d9e0b4135a87c7e96fe823835` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:45.615948 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3bd240-657e-48cb-8fb7-0ed262878a09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0554790d9e0b4135a87c7e96fe823835)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:45.617956 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3bd240-657e-48cb-8fb7-0ed262878a09 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:45.788880 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-8d3bd240-657e-48cb-8fb7-0ed262878a09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0554790d9e0b4135a87c7e96fe823835', 'request_id': 'req-8d3bd240-657e-48cb-8fb7-0ed262878a09'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:45.810847 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-8d3bd240-657e-48cb-8fb7-0ed262878a09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0554790d9e0b4135a87c7e96fe823835', 'request_id': 'req-8d3bd240-657e-48cb-8fb7-0ed262878a09'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:45.855936 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-8d3bd240-657e-48cb-8fb7-0ed262878a09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0554790d9e0b4135a87c7e96fe823835', 'request_id': 'req-8d3bd240-657e-48cb-8fb7-0ed262878a09'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:45.869291 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-8d3bd240-657e-48cb-8fb7-0ed262878a09 admin admin] The token cache is being invalidate because project 0554790d9e0b4135a87c7e96fe823835 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:15:45.869507 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-8d3bd240-657e-48cb-8fb7-0ed262878a09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:45.870355 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-8d3bd240-657e-48cb-8fb7-0ed262878a09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:45.873447 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1187/2397] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:15:45 2022] DELETE /identity/v3/projects/0554790d9e0b4135a87c7e96fe823835 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:15:45.880025 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5074a17c-8a6b-496b-bde2-859fe8ef732a None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:45.880564 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5074a17c-8a6b-496b-bde2-859fe8ef732a None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:45.880690 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5074a17c-8a6b-496b-bde2-859fe8ef732a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:45.903482 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-5074a17c-8a6b-496b-bde2-859fe8ef732a None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:15:45.905594 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-5074a17c-8a6b-496b-bde2-859fe8ef732a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:45.982610 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1211/2398] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:15:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:15:45.992574 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f99b6-d18f-422b-a1cb-e2d84ea26d7c None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:45.992836 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f99b6-d18f-422b-a1cb-e2d84ea26d7c None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:45.993500 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f99b6-d18f-422b-a1cb-e2d84ea26d7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:46.029539 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-564f99b6-d18f-422b-a1cb-e2d84ea26d7c None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:15:46.032248 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-564f99b6-d18f-422b-a1cb-e2d84ea26d7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:46.045068 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1188/2399] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:15:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 03:15:46.050035 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5074a17c-8a6b-496b-bde2-859fe8ef732a None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:46.060675 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5074a17c-8a6b-496b-bde2-859fe8ef732a None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:46.065730 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5f3c9d-ba5b-4ed4-ba1f-97754c303e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:46.066413 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5f3c9d-ba5b-4ed4-ba1f-97754c303e99 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:46.066658 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5f3c9d-ba5b-4ed4-ba1f-97754c303e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:46.066928 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5f3c9d-ba5b-4ed4-ba1f-97754c303e99 admin admin] PATH_INFO: `/v3/domains` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:46.067579 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5f3c9d-ba5b-4ed4-ba1f-97754c303e99 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 03:15:46.067887 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5f3c9d-ba5b-4ed4-ba1f-97754c303e99 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:46.069527 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5f3c9d-ba5b-4ed4-ba1f-97754c303e99 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:46.081845 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1212/2400] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Oct 14 03:15:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 03:15:46.087597 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564f99b6-d18f-422b-a1cb-e2d84ea26d7c None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:46.097913 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564f99b6-d18f-422b-a1cb-e2d84ea26d7c None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:46.103384 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e548e3b2-f658-4a81-8c9d-84b693acc7b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:46.104480 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e548e3b2-f658-4a81-8c9d-84b693acc7b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:46.111834 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e548e3b2-f658-4a81-8c9d-84b693acc7b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:46.111834 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e548e3b2-f658-4a81-8c9d-84b693acc7b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:46.111834 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e548e3b2-f658-4a81-8c9d-84b693acc7b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:46.111834 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e548e3b2-f658-4a81-8c9d-84b693acc7b6 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:46.213220 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1189/2401] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:15:46 2022] POST /identity/v3/projects => generated 373 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:15:46.217391 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5f3c9d-ba5b-4ed4-ba1f-97754c303e99 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:46.228965 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5f3c9d-ba5b-4ed4-ba1f-97754c303e99 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:46.241175 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=100 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:15:46.326179 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb1f78e-d06e-4473-a6bb-da1601a95892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:46.328507 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb1f78e-d06e-4473-a6bb-da1601a95892 admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:46.328507 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb1f78e-d06e-4473-a6bb-da1601a95892 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:46.328507 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb1f78e-d06e-4473-a6bb-da1601a95892 admin admin] PATH_INFO: `/v3/users` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:46.328806 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb1f78e-d06e-4473-a6bb-da1601a95892 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:46.331840 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb1f78e-d06e-4473-a6bb-da1601a95892 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:46.459819 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1213/2402] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:15:46 2022] POST /identity/v3/users => generated 287 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:15:46.463689 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e548e3b2-f658-4a81-8c9d-84b693acc7b6 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:46.480274 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e548e3b2-f658-4a81-8c9d-84b693acc7b6 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:46.489649 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8155dd04-0eb3-458d-85fc-ab33e2616f78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:46.490366 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8155dd04-0eb3-458d-85fc-ab33e2616f78 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:46.490706 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8155dd04-0eb3-458d-85fc-ab33e2616f78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:46.492668 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8155dd04-0eb3-458d-85fc-ab33e2616f78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:46.492668 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8155dd04-0eb3-458d-85fc-ab33e2616f78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:46.493834 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8155dd04-0eb3-458d-85fc-ab33e2616f78 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:46.506816 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1190/2403] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:15:46 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:15:46.510953 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb1f78e-d06e-4473-a6bb-da1601a95892 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:46.524482 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb1f78e-d06e-4473-a6bb-da1601a95892 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:46.530349 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2270a832-45dc-49f1-89d7-208094b4cf29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:46.530965 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2270a832-45dc-49f1-89d7-208094b4cf29 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:46.531254 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2270a832-45dc-49f1-89d7-208094b4cf29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:46.531537 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2270a832-45dc-49f1-89d7-208094b4cf29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:46.532189 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2270a832-45dc-49f1-89d7-208094b4cf29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:46.534560 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2270a832-45dc-49f1-89d7-208094b4cf29 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:46.547304 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1214/2404] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:15:46 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 03:15:46.551351 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8155dd04-0eb3-458d-85fc-ab33e2616f78 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:46.564140 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8155dd04-0eb3-458d-85fc-ab33e2616f78 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:46.569439 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2677f6cc-2470-43e0-b602-fe57c8f89ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:46.570463 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2677f6cc-2470-43e0-b602-fe57c8f89ba3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:46.570596 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2677f6cc-2470-43e0-b602-fe57c8f89ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:46.570865 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2677f6cc-2470-43e0-b602-fe57c8f89ba3 admin admin] PATH_INFO: `/v3/projects/da029b5f99994a6c9a7efb582300100c/users/80db1eed9e594672bbd1eef94a9be0f3/roles/9bd0470b544240209a420f640c79c06c` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:46.587327 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2677f6cc-2470-43e0-b602-fe57c8f89ba3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da029b5f99994a6c9a7efb582300100c, user_id=80db1eed9e594672bbd1eef94a9be0f3, role_id=9bd0470b544240209a420f640c79c06c)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:46.590072 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2677f6cc-2470-43e0-b602-fe57c8f89ba3 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:46.649929 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f049a42-0e85-40c8-a3c9-b4e902ababa4 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:46.650524 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f049a42-0e85-40c8-a3c9-b4e902ababa4 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:46.651035 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f049a42-0e85-40c8-a3c9-b4e902ababa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:46.691808 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1191/2405] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:15:46 2022] PUT /identity/v3/projects/da029b5f99994a6c9a7efb582300100c/users/80db1eed9e594672bbd1eef94a9be0f3/roles/9bd0470b544240209a420f640c79c06c => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:15:46.700144 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-1f049a42-0e85-40c8-a3c9-b4e902ababa4 None None] MFA Rules not processed for user `42efd916cb4b4980a45199d8cc9a4389`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:15:46.703527 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-1f049a42-0e85-40c8-a3c9-b4e902ababa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:46.777644 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f04b3f6-6915-4489-82b8-c57ac5a1b716 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:46.777996 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f04b3f6-6915-4489-82b8-c57ac5a1b716 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:46.778436 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f04b3f6-6915-4489-82b8-c57ac5a1b716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:46.782126 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:15:46.803585 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1215/2406] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:15:46 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:15:46.818522 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-7f04b3f6-6915-4489-82b8-c57ac5a1b716 None None] MFA Rules not processed for user `80db1eed9e594672bbd1eef94a9be0f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:15:46.819863 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-7f04b3f6-6915-4489-82b8-c57ac5a1b716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:46.840366 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:15:46.925060 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1192/2407] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:15:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:15:47.046135 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7824569f-26bc-42bc-8494-3ef97ddaac52 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:47.046602 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7824569f-26bc-42bc-8494-3ef97ddaac52 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:47.046711 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7824569f-26bc-42bc-8494-3ef97ddaac52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:47.049281 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f04b3f6-6915-4489-82b8-c57ac5a1b716 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:47.050916 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-7f04b3f6-6915-4489-82b8-c57ac5a1b716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:47.095501 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-7824569f-26bc-42bc-8494-3ef97ddaac52 None None] MFA Rules not processed for user `41f5223c01ba410abe3539cccb0e9a25`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:15:47.096917 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-7824569f-26bc-42bc-8494-3ef97ddaac52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:47.144509 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f04b3f6-6915-4489-82b8-c57ac5a1b716 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:47.151633 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6668a48d-e026-4d57-9a72-2e3dd00e770b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab599d98b0624a2e89090312bcab713f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:47.152963 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6668a48d-e026-4d57-9a72-2e3dd00e770b service cinder] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:47.153515 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6668a48d-e026-4d57-9a72-2e3dd00e770b service cinder] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:47.154192 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6668a48d-e026-4d57-9a72-2e3dd00e770b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:47.171640 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6668a48d-e026-4d57-9a72-2e3dd00e770b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:47.173860 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6668a48d-e026-4d57-9a72-2e3dd00e770b service cinder] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:47.187534 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1193/2408] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:15:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 03:15:47.208760 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1216/2409] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:15:47 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 03:15:47.314023 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6668a48d-e026-4d57-9a72-2e3dd00e770b service cinder] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:47.324571 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6668a48d-e026-4d57-9a72-2e3dd00e770b service cinder] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:47.327425 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8494796c-8373-4db2-9fde-e54b06f916d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab599d98b0624a2e89090312bcab713f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:47.328037 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8494796c-8373-4db2-9fde-e54b06f916d8 service cinder] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:47.328313 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8494796c-8373-4db2-9fde-e54b06f916d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:47.328631 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8494796c-8373-4db2-9fde-e54b06f916d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:47.330396 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-8494796c-8373-4db2-9fde-e54b06f916d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:47.397248 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8494796c-8373-4db2-9fde-e54b06f916d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:47.398819 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8494796c-8373-4db2-9fde-e54b06f916d8 service cinder] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:47.408388 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1194/2410] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:15:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:15:48.019019 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:15:48.047735 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:15:48.383176 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7824569f-26bc-42bc-8494-3ef97ddaac52 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:48.384617 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-7824569f-26bc-42bc-8494-3ef97ddaac52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:48.453906 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7824569f-26bc-42bc-8494-3ef97ddaac52 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:48.459071 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7506c2e-c84c-4868-992b-121512119845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:48.459852 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7506c2e-c84c-4868-992b-121512119845 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:48.460233 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7506c2e-c84c-4868-992b-121512119845 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:48.460756 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7506c2e-c84c-4868-992b-121512119845 admin admin] PATH_INFO: `/v3/users/840e436082a44317b74b804a12a7fe90` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:48.462075 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7506c2e-c84c-4868-992b-121512119845 admin admin] RBAC: Authorizing `identity:delete_user(user_id=840e436082a44317b74b804a12a7fe90)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:48.464205 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7506c2e-c84c-4868-992b-121512119845 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:48.661480 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-f7506c2e-c84c-4868-992b-121512119845 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '840e436082a44317b74b804a12a7fe90'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:48.662184 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-f7506c2e-c84c-4868-992b-121512119845 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '840e436082a44317b74b804a12a7fe90'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:48.718678 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-f7506c2e-c84c-4868-992b-121512119845 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '840e436082a44317b74b804a12a7fe90'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:48.719377 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-f7506c2e-c84c-4868-992b-121512119845 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '840e436082a44317b74b804a12a7fe90'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:48.728578 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-f7506c2e-c84c-4868-992b-121512119845 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '840e436082a44317b74b804a12a7fe90'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:48.752637 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1217/2411] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:15:48 2022] DELETE /identity/v3/users/840e436082a44317b74b804a12a7fe90 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:15:48.757330 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1d5022-ddab-4f95-91e9-3ddbf40881cc None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:48.757775 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1d5022-ddab-4f95-91e9-3ddbf40881cc None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:48.758225 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1d5022-ddab-4f95-91e9-3ddbf40881cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:48.779003 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-5d1d5022-ddab-4f95-91e9-3ddbf40881cc None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:15:48.780241 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-5d1d5022-ddab-4f95-91e9-3ddbf40881cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:48.838673 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1195/2412] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:15:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:15:48.849271 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7506c2e-c84c-4868-992b-121512119845 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:48.850925 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-f7506c2e-c84c-4868-992b-121512119845 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:48.906671 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7506c2e-c84c-4868-992b-121512119845 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:48.910896 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44aef03-bdb4-4cb8-b5b9-fbd04a61539a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:48.911498 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44aef03-bdb4-4cb8-b5b9-fbd04a61539a service neutron] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:48.911777 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44aef03-bdb4-4cb8-b5b9-fbd04a61539a service neutron] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:48.912049 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44aef03-bdb4-4cb8-b5b9-fbd04a61539a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:48.921212 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44aef03-bdb4-4cb8-b5b9-fbd04a61539a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:48.922679 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44aef03-bdb4-4cb8-b5b9-fbd04a61539a service neutron] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:48.932216 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1218/2413] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:15:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:15:49.107872 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:INSERT=1,keystone:SELECT=98,keystone:DELETE=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:15:49.325753 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1d5022-ddab-4f95-91e9-3ddbf40881cc None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:49.327208 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-5d1d5022-ddab-4f95-91e9-3ddbf40881cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:49.372513 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44aef03-bdb4-4cb8-b5b9-fbd04a61539a service neutron] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:49.374931 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-c44aef03-bdb4-4cb8-b5b9-fbd04a61539a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:49.391454 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44aef03-bdb4-4cb8-b5b9-fbd04a61539a service neutron] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:49.410897 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c55e892-eafb-43ce-868a-991e7529ca25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:49.413881 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c55e892-eafb-43ce-868a-991e7529ca25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:49.414287 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c55e892-eafb-43ce-868a-991e7529ca25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:49.414605 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c55e892-eafb-43ce-868a-991e7529ca25 admin admin] PATH_INFO: `/v3/projects/dd17139d63c444beaed77f54bd4ff033` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:49.415967 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c55e892-eafb-43ce-868a-991e7529ca25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd17139d63c444beaed77f54bd4ff033)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:49.416594 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1d5022-ddab-4f95-91e9-3ddbf40881cc None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:49.418159 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c55e892-eafb-43ce-868a-991e7529ca25 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:49.420313 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061ece96-7c66-4eb8-9f72-7b203df6e6e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab599d98b0624a2e89090312bcab713f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:49.420959 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ece96-7c66-4eb8-9f72-7b203df6e6e7 service cinder] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:49.421237 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ece96-7c66-4eb8-9f72-7b203df6e6e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:49.421535 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ece96-7c66-4eb8-9f72-7b203df6e6e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:49.423436 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-061ece96-7c66-4eb8-9f72-7b203df6e6e7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:49.468449 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:15:49.489875 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061ece96-7c66-4eb8-9f72-7b203df6e6e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:49.491418 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061ece96-7c66-4eb8-9f72-7b203df6e6e7 service cinder] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:49.501838 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1196/2414] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:15:49 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:15:49.561171 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-6c55e892-eafb-43ce-868a-991e7529ca25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd17139d63c444beaed77f54bd4ff033', 'request_id': 'req-6c55e892-eafb-43ce-868a-991e7529ca25'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:49.567785 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-6c55e892-eafb-43ce-868a-991e7529ca25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd17139d63c444beaed77f54bd4ff033', 'request_id': 'req-6c55e892-eafb-43ce-868a-991e7529ca25'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:49.653286 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-6c55e892-eafb-43ce-868a-991e7529ca25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd17139d63c444beaed77f54bd4ff033', 'request_id': 'req-6c55e892-eafb-43ce-868a-991e7529ca25'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:49.668129 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-6c55e892-eafb-43ce-868a-991e7529ca25 admin admin] The token cache is being invalidate because project dd17139d63c444beaed77f54bd4ff033 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81517) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:15:49.668430 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-6c55e892-eafb-43ce-868a-991e7529ca25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:49.669215 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-6c55e892-eafb-43ce-868a-991e7529ca25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:49.671494 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1219/2415] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:15:49 2022] DELETE /identity/v3/projects/dd17139d63c444beaed77f54bd4ff033 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 03:15:49.675394 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061ece96-7c66-4eb8-9f72-7b203df6e6e7 service cinder] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:49.683961 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:15:49.697336 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061ece96-7c66-4eb8-9f72-7b203df6e6e7 service cinder] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:49.759195 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d868c736-1d9e-4340-8670-60473f1b5a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:49.759845 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d868c736-1d9e-4340-8670-60473f1b5a64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:49.760172 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d868c736-1d9e-4340-8670-60473f1b5a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:49.760447 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d868c736-1d9e-4340-8670-60473f1b5a64 admin admin] PATH_INFO: `/v3/users/bc5389de37cd43e383e763ccb7342dd2` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:49.761600 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d868c736-1d9e-4340-8670-60473f1b5a64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc5389de37cd43e383e763ccb7342dd2)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:49.763614 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d868c736-1d9e-4340-8670-60473f1b5a64 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:50.041975 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-d868c736-1d9e-4340-8670-60473f1b5a64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc5389de37cd43e383e763ccb7342dd2'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:50.052634 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-d868c736-1d9e-4340-8670-60473f1b5a64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc5389de37cd43e383e763ccb7342dd2'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:50.066327 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-d868c736-1d9e-4340-8670-60473f1b5a64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc5389de37cd43e383e763ccb7342dd2'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:50.067086 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-d868c736-1d9e-4340-8670-60473f1b5a64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc5389de37cd43e383e763ccb7342dd2'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:50.135995 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-d868c736-1d9e-4340-8670-60473f1b5a64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc5389de37cd43e383e763ccb7342dd2'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:50.142401 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1197/2416] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:15:49 2022] DELETE /identity/v3/users/bc5389de37cd43e383e763ccb7342dd2 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 03:15:50.382161 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:15:50.604233 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c55e892-eafb-43ce-868a-991e7529ca25 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:50.605738 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-6c55e892-eafb-43ce-868a-991e7529ca25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:50.622728 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c55e892-eafb-43ce-868a-991e7529ca25 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:50.637156 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:15:50.681508 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9591bd1a-0365-450f-828f-fcc1b3971a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:50.682097 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9591bd1a-0365-450f-828f-fcc1b3971a29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:50.682378 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9591bd1a-0365-450f-828f-fcc1b3971a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:50.682648 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9591bd1a-0365-450f-828f-fcc1b3971a29 admin admin] PATH_INFO: `/v3/projects/73319bfb27454f8c93514f7cb0a6de67` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:50.683757 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9591bd1a-0365-450f-828f-fcc1b3971a29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73319bfb27454f8c93514f7cb0a6de67)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:50.687722 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9591bd1a-0365-450f-828f-fcc1b3971a29 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:50.804536 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-9591bd1a-0365-450f-828f-fcc1b3971a29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73319bfb27454f8c93514f7cb0a6de67', 'request_id': 'req-9591bd1a-0365-450f-828f-fcc1b3971a29'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:50.810513 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-9591bd1a-0365-450f-828f-fcc1b3971a29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73319bfb27454f8c93514f7cb0a6de67', 'request_id': 'req-9591bd1a-0365-450f-828f-fcc1b3971a29'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:50.867261 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-9591bd1a-0365-450f-828f-fcc1b3971a29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73319bfb27454f8c93514f7cb0a6de67', 'request_id': 'req-9591bd1a-0365-450f-828f-fcc1b3971a29'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:50.877977 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-9591bd1a-0365-450f-828f-fcc1b3971a29 admin admin] The token cache is being invalidate because project 73319bfb27454f8c93514f7cb0a6de67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81517) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:15:50.878271 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-9591bd1a-0365-450f-828f-fcc1b3971a29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:50.878912 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-9591bd1a-0365-450f-828f-fcc1b3971a29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:50.880387 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1220/2417] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:15:50 2022] DELETE /identity/v3/projects/73319bfb27454f8c93514f7cb0a6de67 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:15:50.884462 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d868c736-1d9e-4340-8670-60473f1b5a64 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:50.885905 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-d868c736-1d9e-4340-8670-60473f1b5a64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:50.899314 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d868c736-1d9e-4340-8670-60473f1b5a64 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:50.930507 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:15:50.959602 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fe084c-3575-46ff-bcbb-83dfba9cb038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:50.960152 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fe084c-3575-46ff-bcbb-83dfba9cb038 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:50.960435 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fe084c-3575-46ff-bcbb-83dfba9cb038 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:50.960705 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fe084c-3575-46ff-bcbb-83dfba9cb038 admin admin] PATH_INFO: `/v3/users/942747c1d4f94560be4afaeac55dd78c` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:50.961879 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fe084c-3575-46ff-bcbb-83dfba9cb038 admin admin] RBAC: Authorizing `identity:delete_user(user_id=942747c1d4f94560be4afaeac55dd78c)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:50.963785 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fe084c-3575-46ff-bcbb-83dfba9cb038 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:51.148511 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-e3fe084c-3575-46ff-bcbb-83dfba9cb038 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '942747c1d4f94560be4afaeac55dd78c'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:51.156032 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-e3fe084c-3575-46ff-bcbb-83dfba9cb038 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '942747c1d4f94560be4afaeac55dd78c'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:51.169423 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-e3fe084c-3575-46ff-bcbb-83dfba9cb038 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '942747c1d4f94560be4afaeac55dd78c'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:51.170139 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-e3fe084c-3575-46ff-bcbb-83dfba9cb038 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '942747c1d4f94560be4afaeac55dd78c'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:51.267929 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-e3fe084c-3575-46ff-bcbb-83dfba9cb038 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '942747c1d4f94560be4afaeac55dd78c'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:51.275403 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1198/2418] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:15:50 2022] DELETE /identity/v3/users/942747c1d4f94560be4afaeac55dd78c => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 03:15:51.672679 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9591bd1a-0365-450f-828f-fcc1b3971a29 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:51.674195 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-9591bd1a-0365-450f-828f-fcc1b3971a29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:51.697470 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9591bd1a-0365-450f-828f-fcc1b3971a29 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:51.761415 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f40c4f-4c18-4378-8cb7-fde27b8b02dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:51.762185 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f40c4f-4c18-4378-8cb7-fde27b8b02dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:51.762537 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f40c4f-4c18-4378-8cb7-fde27b8b02dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:51.762947 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f40c4f-4c18-4378-8cb7-fde27b8b02dd admin admin] PATH_INFO: `/v3/projects/c2fdf23f0560401ab8740dd8b024df6b` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:51.764241 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f40c4f-4c18-4378-8cb7-fde27b8b02dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2fdf23f0560401ab8740dd8b024df6b)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:51.766278 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f40c4f-4c18-4378-8cb7-fde27b8b02dd admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:51.881784 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-b2f40c4f-4c18-4378-8cb7-fde27b8b02dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2fdf23f0560401ab8740dd8b024df6b', 'request_id': 'req-b2f40c4f-4c18-4378-8cb7-fde27b8b02dd'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:51.881784 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:15:51.882417 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-b2f40c4f-4c18-4378-8cb7-fde27b8b02dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2fdf23f0560401ab8740dd8b024df6b', 'request_id': 'req-b2f40c4f-4c18-4378-8cb7-fde27b8b02dd'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:51.981585 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-b2f40c4f-4c18-4378-8cb7-fde27b8b02dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2fdf23f0560401ab8740dd8b024df6b', 'request_id': 'req-b2f40c4f-4c18-4378-8cb7-fde27b8b02dd'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:51.993509 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-b2f40c4f-4c18-4378-8cb7-fde27b8b02dd admin admin] The token cache is being invalidate because project c2fdf23f0560401ab8740dd8b024df6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81517) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:15:51.993851 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-b2f40c4f-4c18-4378-8cb7-fde27b8b02dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:51.994507 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-b2f40c4f-4c18-4378-8cb7-fde27b8b02dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:51.995991 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1221/2419] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:15:51 2022] DELETE /identity/v3/projects/c2fdf23f0560401ab8740dd8b024df6b => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:15:52.007523 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab1b3a-7ba0-4cab-8cf2-97c2dd696bef None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:52.008046 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab1b3a-7ba0-4cab-8cf2-97c2dd696bef None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:52.008278 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab1b3a-7ba0-4cab-8cf2-97c2dd696bef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:52.031105 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-38ab1b3a-7ba0-4cab-8cf2-97c2dd696bef None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:15:52.032297 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-38ab1b3a-7ba0-4cab-8cf2-97c2dd696bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:52.093691 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1199/2420] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:15:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:15:52.102320 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e15fcd-8986-4248-95bc-2971dde50b10 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:52.102627 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e15fcd-8986-4248-95bc-2971dde50b10 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:52.103011 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e15fcd-8986-4248-95bc-2971dde50b10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:52.125241 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-d0e15fcd-8986-4248-95bc-2971dde50b10 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:15:52.126690 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-d0e15fcd-8986-4248-95bc-2971dde50b10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:52.135293 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1222/2421] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:15:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:15:52.139563 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ab1b3a-7ba0-4cab-8cf2-97c2dd696bef None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:52.149125 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ab1b3a-7ba0-4cab-8cf2-97c2dd696bef None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:52.153722 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ddfaf9-689c-4e67-8a16-05e46c4da491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:52.154351 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ddfaf9-689c-4e67-8a16-05e46c4da491 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:52.154627 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ddfaf9-689c-4e67-8a16-05e46c4da491 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:52.154921 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ddfaf9-689c-4e67-8a16-05e46c4da491 admin admin] PATH_INFO: `/v3/domains` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:52.155557 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ddfaf9-689c-4e67-8a16-05e46c4da491 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 03:15:52.155857 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ddfaf9-689c-4e67-8a16-05e46c4da491 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:52.157440 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ddfaf9-689c-4e67-8a16-05e46c4da491 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:52.169409 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1200/2422] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Oct 14 03:15:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 03:15:52.173715 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e15fcd-8986-4248-95bc-2971dde50b10 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:52.184110 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e15fcd-8986-4248-95bc-2971dde50b10 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:52.188911 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b1bfd7-2b30-41e4-9906-494fffe614b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:52.189519 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b1bfd7-2b30-41e4-9906-494fffe614b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:52.189834 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b1bfd7-2b30-41e4-9906-494fffe614b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:52.190114 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b1bfd7-2b30-41e4-9906-494fffe614b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:52.190872 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b1bfd7-2b30-41e4-9906-494fffe614b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:52.192681 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b1bfd7-2b30-41e4-9906-494fffe614b4 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:52.292352 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1223/2423] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:15:52 2022] POST /identity/v3/projects => generated 367 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:15:52.296582 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ddfaf9-689c-4e67-8a16-05e46c4da491 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:52.308834 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ddfaf9-689c-4e67-8a16-05e46c4da491 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:52.341629 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:15:52.364561 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2c31fe-698b-4adb-b4cd-c744c50709c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:52.365217 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2c31fe-698b-4adb-b4cd-c744c50709c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:52.365604 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2c31fe-698b-4adb-b4cd-c744c50709c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:52.366099 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2c31fe-698b-4adb-b4cd-c744c50709c9 admin admin] PATH_INFO: `/v3/users` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:52.366736 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2c31fe-698b-4adb-b4cd-c744c50709c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:52.368639 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2c31fe-698b-4adb-b4cd-c744c50709c9 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:52.459883 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1201/2424] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:15:52 2022] POST /identity/v3/users => generated 284 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:15:52.464039 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b1bfd7-2b30-41e4-9906-494fffe614b4 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:52.475492 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b1bfd7-2b30-41e4-9906-494fffe614b4 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:52.480208 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0818942-cc01-4803-9c16-388368538df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:52.480873 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0818942-cc01-4803-9c16-388368538df5 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:52.481224 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0818942-cc01-4803-9c16-388368538df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:52.481569 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0818942-cc01-4803-9c16-388368538df5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:52.482316 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0818942-cc01-4803-9c16-388368538df5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:52.484172 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0818942-cc01-4803-9c16-388368538df5 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:52.494517 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1224/2425] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:15:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:15:52.498673 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2c31fe-698b-4adb-b4cd-c744c50709c9 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:52.509113 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2c31fe-698b-4adb-b4cd-c744c50709c9 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:52.514031 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bbfa57-c84a-4d94-9887-354213fed1fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:52.517711 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bbfa57-c84a-4d94-9887-354213fed1fa admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:52.517711 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bbfa57-c84a-4d94-9887-354213fed1fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:52.517711 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bbfa57-c84a-4d94-9887-354213fed1fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:52.517711 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bbfa57-c84a-4d94-9887-354213fed1fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:52.517711 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bbfa57-c84a-4d94-9887-354213fed1fa admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:52.528254 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1202/2426] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:15:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:15:52.532160 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0818942-cc01-4803-9c16-388368538df5 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:52.542525 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0818942-cc01-4803-9c16-388368538df5 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:52.547079 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e803971-158a-4dff-b9fd-bffe90dcb9de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:52.547713 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e803971-158a-4dff-b9fd-bffe90dcb9de admin admin] REQUEST_METHOD: `PUT` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:52.547999 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e803971-158a-4dff-b9fd-bffe90dcb9de admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:52.548269 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e803971-158a-4dff-b9fd-bffe90dcb9de admin admin] PATH_INFO: `/v3/projects/b3bd42abc89e4835935cf6cb07a668b9/users/0f745e320f8c4cd5b1772054dc3c03cf/roles/9bd0470b544240209a420f640c79c06c` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:52.563487 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e803971-158a-4dff-b9fd-bffe90dcb9de admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3bd42abc89e4835935cf6cb07a668b9, user_id=0f745e320f8c4cd5b1772054dc3c03cf, role_id=9bd0470b544240209a420f640c79c06c)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:52.566101 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e803971-158a-4dff-b9fd-bffe90dcb9de admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:52.710356 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1225/2427] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:15:52 2022] PUT /identity/v3/projects/b3bd42abc89e4835935cf6cb07a668b9/users/0f745e320f8c4cd5b1772054dc3c03cf/roles/9bd0470b544240209a420f640c79c06c => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:15:52.721030 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203badba-5368-43e8-a2e6-2391aa7e52b1 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:52.721317 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203badba-5368-43e8-a2e6-2391aa7e52b1 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:52.721590 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203badba-5368-43e8-a2e6-2391aa7e52b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:52.750832 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-203badba-5368-43e8-a2e6-2391aa7e52b1 None None] MFA Rules not processed for user `0f745e320f8c4cd5b1772054dc3c03cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:15:52.752038 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-203badba-5368-43e8-a2e6-2391aa7e52b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:52.819541 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1203/2428] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:15:52 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:15:52.833464 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e803971-158a-4dff-b9fd-bffe90dcb9de admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:52.835316 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-2e803971-158a-4dff-b9fd-bffe90dcb9de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:52.898633 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e803971-158a-4dff-b9fd-bffe90dcb9de admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:52.902343 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb2929c-1d3c-4954-aeb6-eaf36e8bd211 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a3cd1fbe6df48f8aa90d4ea69905e9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:52.902938 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb2929c-1d3c-4954-aeb6-eaf36e8bd211 service glance] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:52.903217 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb2929c-1d3c-4954-aeb6-eaf36e8bd211 service glance] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:52.903490 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb2929c-1d3c-4954-aeb6-eaf36e8bd211 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:52.912178 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb2929c-1d3c-4954-aeb6-eaf36e8bd211 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:52.913626 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb2929c-1d3c-4954-aeb6-eaf36e8bd211 service glance] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:52.923036 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1226/2429] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:15:52 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:15:59.632326 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203badba-5368-43e8-a2e6-2391aa7e52b1 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:15:59.633767 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-203badba-5368-43e8-a2e6-2391aa7e52b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:15:59.698626 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203badba-5368-43e8-a2e6-2391aa7e52b1 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:15:59.703279 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803ccf1e-38f1-4527-888a-9522d2cf87ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:15:59.703914 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803ccf1e-38f1-4527-888a-9522d2cf87ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:15:59.704201 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803ccf1e-38f1-4527-888a-9522d2cf87ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:15:59.704477 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803ccf1e-38f1-4527-888a-9522d2cf87ee admin admin] PATH_INFO: `/v3/users/80db1eed9e594672bbd1eef94a9be0f3` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:15:59.705622 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803ccf1e-38f1-4527-888a-9522d2cf87ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=80db1eed9e594672bbd1eef94a9be0f3)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:15:59.707617 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803ccf1e-38f1-4527-888a-9522d2cf87ee admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:15:59.932580 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-803ccf1e-38f1-4527-888a-9522d2cf87ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80db1eed9e594672bbd1eef94a9be0f3'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:59.933854 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:15:59.940171 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-803ccf1e-38f1-4527-888a-9522d2cf87ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80db1eed9e594672bbd1eef94a9be0f3'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:59.952433 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-803ccf1e-38f1-4527-888a-9522d2cf87ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80db1eed9e594672bbd1eef94a9be0f3'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:15:59.953326 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-803ccf1e-38f1-4527-888a-9522d2cf87ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80db1eed9e594672bbd1eef94a9be0f3'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:00.030045 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-803ccf1e-38f1-4527-888a-9522d2cf87ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80db1eed9e594672bbd1eef94a9be0f3'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:00.037326 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1204/2430] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:15:59 2022] DELETE /identity/v3/users/80db1eed9e594672bbd1eef94a9be0f3 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:16:00.042376 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a683090a-e260-4d63-8028-0e23d850adfa None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:00.042913 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a683090a-e260-4d63-8028-0e23d850adfa None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:00.042913 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a683090a-e260-4d63-8028-0e23d850adfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:00.061802 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-a683090a-e260-4d63-8028-0e23d850adfa None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:16:00.062942 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-a683090a-e260-4d63-8028-0e23d850adfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:16:00.125387 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1227/2431] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:16:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:16:00.140510 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803ccf1e-38f1-4527-888a-9522d2cf87ee admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:16:00.142151 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-803ccf1e-38f1-4527-888a-9522d2cf87ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:16:00.207119 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803ccf1e-38f1-4527-888a-9522d2cf87ee admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:16:00.211757 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d719ce-2220-4622-a699-2a7f25a5b55f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:16:00.212743 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d719ce-2220-4622-a699-2a7f25a5b55f service neutron] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:00.213028 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d719ce-2220-4622-a699-2a7f25a5b55f service neutron] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:00.213398 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d719ce-2220-4622-a699-2a7f25a5b55f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:00.224254 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d719ce-2220-4622-a699-2a7f25a5b55f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:16:00.225981 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d719ce-2220-4622-a699-2a7f25a5b55f service neutron] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:16:00.236590 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1205/2432] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:16:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:16:00.671508 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a683090a-e260-4d63-8028-0e23d850adfa None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:16:00.673116 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-a683090a-e260-4d63-8028-0e23d850adfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:16:00.692402 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a683090a-e260-4d63-8028-0e23d850adfa None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:16:00.699275 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac6b682-1296-4395-9643-45fd0d2b4b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:16:00.700033 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac6b682-1296-4395-9643-45fd0d2b4b05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:00.700214 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac6b682-1296-4395-9643-45fd0d2b4b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:00.700483 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac6b682-1296-4395-9643-45fd0d2b4b05 admin admin] PATH_INFO: `/v3/projects/da029b5f99994a6c9a7efb582300100c` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:00.701723 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac6b682-1296-4395-9643-45fd0d2b4b05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da029b5f99994a6c9a7efb582300100c)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:16:00.708196 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac6b682-1296-4395-9643-45fd0d2b4b05 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:16:00.716882 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:16:00.817405 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-bac6b682-1296-4395-9643-45fd0d2b4b05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da029b5f99994a6c9a7efb582300100c', 'request_id': 'req-bac6b682-1296-4395-9643-45fd0d2b4b05'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:00.824716 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-bac6b682-1296-4395-9643-45fd0d2b4b05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da029b5f99994a6c9a7efb582300100c', 'request_id': 'req-bac6b682-1296-4395-9643-45fd0d2b4b05'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:00.896397 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-bac6b682-1296-4395-9643-45fd0d2b4b05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da029b5f99994a6c9a7efb582300100c', 'request_id': 'req-bac6b682-1296-4395-9643-45fd0d2b4b05'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:00.907827 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-bac6b682-1296-4395-9643-45fd0d2b4b05 admin admin] The token cache is being invalidate because project da029b5f99994a6c9a7efb582300100c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81517) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:16:00.908120 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-bac6b682-1296-4395-9643-45fd0d2b4b05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:00.908767 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-bac6b682-1296-4395-9643-45fd0d2b4b05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:00.910421 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1228/2433] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:16:00 2022] DELETE /identity/v3/projects/da029b5f99994a6c9a7efb582300100c => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:16:05.982738 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d719ce-2220-4622-a699-2a7f25a5b55f service neutron] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:16:05.984186 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-19d719ce-2220-4622-a699-2a7f25a5b55f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:16:05.998654 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d719ce-2220-4622-a699-2a7f25a5b55f service neutron] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:16:06.059365 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d259be-c04c-4536-b789-0d9604c2e11f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:16:06.060083 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d259be-c04c-4536-b789-0d9604c2e11f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:06.060396 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d259be-c04c-4536-b789-0d9604c2e11f admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:06.060689 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d259be-c04c-4536-b789-0d9604c2e11f admin admin] PATH_INFO: `/v3/users/440a43fbd0aa4645895da5057bd41870` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:06.061923 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d259be-c04c-4536-b789-0d9604c2e11f admin admin] RBAC: Authorizing `identity:delete_user(user_id=440a43fbd0aa4645895da5057bd41870)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:16:06.063907 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d259be-c04c-4536-b789-0d9604c2e11f admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:16:06.217945 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-b3d259be-c04c-4536-b789-0d9604c2e11f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '440a43fbd0aa4645895da5057bd41870'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:06.230588 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-b3d259be-c04c-4536-b789-0d9604c2e11f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '440a43fbd0aa4645895da5057bd41870'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:06.237116 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:16:06.240525 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-b3d259be-c04c-4536-b789-0d9604c2e11f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '440a43fbd0aa4645895da5057bd41870'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:06.241287 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-b3d259be-c04c-4536-b789-0d9604c2e11f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '440a43fbd0aa4645895da5057bd41870'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:06.323879 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-b3d259be-c04c-4536-b789-0d9604c2e11f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '440a43fbd0aa4645895da5057bd41870'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:06.330588 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1206/2434] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:16:05 2022] DELETE /identity/v3/users/440a43fbd0aa4645895da5057bd41870 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:16:06.583369 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac6b682-1296-4395-9643-45fd0d2b4b05 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:16:06.585525 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-bac6b682-1296-4395-9643-45fd0d2b4b05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:16:06.600752 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac6b682-1296-4395-9643-45fd0d2b4b05 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:16:06.661159 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eae7962-baa4-40a5-ba40-e164dc82cdd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:16:06.661824 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eae7962-baa4-40a5-ba40-e164dc82cdd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:06.662164 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eae7962-baa4-40a5-ba40-e164dc82cdd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:06.662443 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eae7962-baa4-40a5-ba40-e164dc82cdd9 admin admin] PATH_INFO: `/v3/projects/f7f11955b8c9406daa11fe9576edef04` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:06.663552 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eae7962-baa4-40a5-ba40-e164dc82cdd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7f11955b8c9406daa11fe9576edef04)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:16:06.666131 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eae7962-baa4-40a5-ba40-e164dc82cdd9 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:16:06.769438 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-2eae7962-baa4-40a5-ba40-e164dc82cdd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7f11955b8c9406daa11fe9576edef04', 'request_id': 'req-2eae7962-baa4-40a5-ba40-e164dc82cdd9'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:06.775953 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-2eae7962-baa4-40a5-ba40-e164dc82cdd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7f11955b8c9406daa11fe9576edef04', 'request_id': 'req-2eae7962-baa4-40a5-ba40-e164dc82cdd9'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:06.884584 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-2eae7962-baa4-40a5-ba40-e164dc82cdd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7f11955b8c9406daa11fe9576edef04', 'request_id': 'req-2eae7962-baa4-40a5-ba40-e164dc82cdd9'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:06.896093 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-2eae7962-baa4-40a5-ba40-e164dc82cdd9 admin admin] The token cache is being invalidate because project f7f11955b8c9406daa11fe9576edef04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81517) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:16:06.896398 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-2eae7962-baa4-40a5-ba40-e164dc82cdd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:06.897002 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-2eae7962-baa4-40a5-ba40-e164dc82cdd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:06.898563 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1229/2435] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:16:06 2022] DELETE /identity/v3/projects/f7f11955b8c9406daa11fe9576edef04 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:16:06.903243 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d259be-c04c-4536-b789-0d9604c2e11f admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:16:06.904687 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-b3d259be-c04c-4536-b789-0d9604c2e11f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:16:06.917595 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d259be-c04c-4536-b789-0d9604c2e11f admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:16:06.974597 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67023ee0-baf7-4977-b8f6-f90b8076a67b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:16:06.975239 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67023ee0-baf7-4977-b8f6-f90b8076a67b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:06.975517 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67023ee0-baf7-4977-b8f6-f90b8076a67b admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:06.975791 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67023ee0-baf7-4977-b8f6-f90b8076a67b admin admin] PATH_INFO: `/v3/users/41f5223c01ba410abe3539cccb0e9a25` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:06.976931 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67023ee0-baf7-4977-b8f6-f90b8076a67b admin admin] RBAC: Authorizing `identity:delete_user(user_id=41f5223c01ba410abe3539cccb0e9a25)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:16:06.978942 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67023ee0-baf7-4977-b8f6-f90b8076a67b admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:16:07.178051 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-67023ee0-baf7-4977-b8f6-f90b8076a67b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41f5223c01ba410abe3539cccb0e9a25'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:07.185653 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-67023ee0-baf7-4977-b8f6-f90b8076a67b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41f5223c01ba410abe3539cccb0e9a25'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:07.198207 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-67023ee0-baf7-4977-b8f6-f90b8076a67b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41f5223c01ba410abe3539cccb0e9a25'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:07.198974 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-67023ee0-baf7-4977-b8f6-f90b8076a67b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41f5223c01ba410abe3539cccb0e9a25'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:07.269512 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-67023ee0-baf7-4977-b8f6-f90b8076a67b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41f5223c01ba410abe3539cccb0e9a25'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:07.275950 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1207/2436] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:16:06 2022] DELETE /identity/v3/users/41f5223c01ba410abe3539cccb0e9a25 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:16:07.501160 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eae7962-baa4-40a5-ba40-e164dc82cdd9 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:16:07.502919 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-2eae7962-baa4-40a5-ba40-e164dc82cdd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:16:07.531024 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eae7962-baa4-40a5-ba40-e164dc82cdd9 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:16:07.560928 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:16:07.589277 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d173b911-0bce-490e-93d3-433ad1541e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:16:07.590086 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d173b911-0bce-490e-93d3-433ad1541e9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:07.590741 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d173b911-0bce-490e-93d3-433ad1541e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:07.591073 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d173b911-0bce-490e-93d3-433ad1541e9f admin admin] PATH_INFO: `/v3/projects/ec85ee91ec734d0dbd86da9a12f44545` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:07.592292 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d173b911-0bce-490e-93d3-433ad1541e9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec85ee91ec734d0dbd86da9a12f44545)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:16:07.594272 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d173b911-0bce-490e-93d3-433ad1541e9f admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:16:07.704143 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-d173b911-0bce-490e-93d3-433ad1541e9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec85ee91ec734d0dbd86da9a12f44545', 'request_id': 'req-d173b911-0bce-490e-93d3-433ad1541e9f'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:07.710878 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-d173b911-0bce-490e-93d3-433ad1541e9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec85ee91ec734d0dbd86da9a12f44545', 'request_id': 'req-d173b911-0bce-490e-93d3-433ad1541e9f'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:07.790198 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-d173b911-0bce-490e-93d3-433ad1541e9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec85ee91ec734d0dbd86da9a12f44545', 'request_id': 'req-d173b911-0bce-490e-93d3-433ad1541e9f'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:07.801448 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-d173b911-0bce-490e-93d3-433ad1541e9f admin admin] The token cache is being invalidate because project ec85ee91ec734d0dbd86da9a12f44545 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81517) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:16:07.801741 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-d173b911-0bce-490e-93d3-433ad1541e9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:07.802540 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-d173b911-0bce-490e-93d3-433ad1541e9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:07.804710 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1230/2437] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:16:07 2022] DELETE /identity/v3/projects/ec85ee91ec734d0dbd86da9a12f44545 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:16:07.810116 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2725f2b5-554b-4819-9797-036e60d3f00d None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:07.810412 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2725f2b5-554b-4819-9797-036e60d3f00d None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:07.810727 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2725f2b5-554b-4819-9797-036e60d3f00d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:07.831129 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-2725f2b5-554b-4819-9797-036e60d3f00d None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:16:07.832349 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-2725f2b5-554b-4819-9797-036e60d3f00d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:16:07.891191 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1208/2438] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:16:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:16:07.899219 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfffb08-e3d5-4d8f-ba4d-c72aa03e1c12 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:07.899510 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfffb08-e3d5-4d8f-ba4d-c72aa03e1c12 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:07.899821 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfffb08-e3d5-4d8f-ba4d-c72aa03e1c12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:07.920486 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-3dfffb08-e3d5-4d8f-ba4d-c72aa03e1c12 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:16:07.921800 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-3dfffb08-e3d5-4d8f-ba4d-c72aa03e1c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:16:07.929730 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1231/2439] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:16:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:16:07.934604 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2725f2b5-554b-4819-9797-036e60d3f00d None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:16:07.944504 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2725f2b5-554b-4819-9797-036e60d3f00d None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:16:07.949071 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2beed5-e9d2-4435-84ee-cfd42fecb581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:16:07.949656 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2beed5-e9d2-4435-84ee-cfd42fecb581 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:07.949983 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2beed5-e9d2-4435-84ee-cfd42fecb581 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:07.950257 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2beed5-e9d2-4435-84ee-cfd42fecb581 admin admin] PATH_INFO: `/v3/domains` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:07.950898 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2beed5-e9d2-4435-84ee-cfd42fecb581 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 03:16:07.951235 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2beed5-e9d2-4435-84ee-cfd42fecb581 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:16:07.952858 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2beed5-e9d2-4435-84ee-cfd42fecb581 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:16:07.954391 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:16:07.965694 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1209/2440] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Oct 14 03:16:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 03:16:07.969867 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfffb08-e3d5-4d8f-ba4d-c72aa03e1c12 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:16:07.980022 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfffb08-e3d5-4d8f-ba4d-c72aa03e1c12 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:16:07.985067 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a62686-b7fa-4881-bd92-6a0ead03e9b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:16:07.985681 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a62686-b7fa-4881-bd92-6a0ead03e9b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:07.985994 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a62686-b7fa-4881-bd92-6a0ead03e9b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:07.986273 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a62686-b7fa-4881-bd92-6a0ead03e9b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:07.986953 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a62686-b7fa-4881-bd92-6a0ead03e9b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:16:07.989196 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a62686-b7fa-4881-bd92-6a0ead03e9b4 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:16:08.080229 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1232/2441] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:16:07 2022] POST /identity/v3/projects => generated 385 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:16:08.084451 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2beed5-e9d2-4435-84ee-cfd42fecb581 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:16:08.095608 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2beed5-e9d2-4435-84ee-cfd42fecb581 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:16:08.152473 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcea8165-198e-43cd-bac0-484803155dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:16:08.153083 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcea8165-198e-43cd-bac0-484803155dc7 admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:08.153372 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcea8165-198e-43cd-bac0-484803155dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:08.153647 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcea8165-198e-43cd-bac0-484803155dc7 admin admin] PATH_INFO: `/v3/users` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:08.154337 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcea8165-198e-43cd-bac0-484803155dc7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:16:08.156255 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcea8165-198e-43cd-bac0-484803155dc7 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:16:08.256299 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1210/2442] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:16:08 2022] POST /identity/v3/users => generated 293 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:16:08.260379 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a62686-b7fa-4881-bd92-6a0ead03e9b4 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:16:08.271035 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a62686-b7fa-4881-bd92-6a0ead03e9b4 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:16:08.276325 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67198dd2-933d-4056-8900-c16dd7c549ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:16:08.276966 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67198dd2-933d-4056-8900-c16dd7c549ea admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:08.277336 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67198dd2-933d-4056-8900-c16dd7c549ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:08.277622 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67198dd2-933d-4056-8900-c16dd7c549ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:08.278318 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67198dd2-933d-4056-8900-c16dd7c549ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:16:08.279926 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67198dd2-933d-4056-8900-c16dd7c549ea admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:16:08.290805 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1233/2443] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:16:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:16:08.298831 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcea8165-198e-43cd-bac0-484803155dc7 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:16:08.308971 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcea8165-198e-43cd-bac0-484803155dc7 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:16:08.313663 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4b8a3c-744b-418d-b05e-6331b2d0339f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:16:08.314286 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4b8a3c-744b-418d-b05e-6331b2d0339f admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:08.314603 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4b8a3c-744b-418d-b05e-6331b2d0339f admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:08.314845 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4b8a3c-744b-418d-b05e-6331b2d0339f admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:08.315523 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4b8a3c-744b-418d-b05e-6331b2d0339f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:16:08.317115 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4b8a3c-744b-418d-b05e-6331b2d0339f admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:16:08.328734 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1211/2444] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:16:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:16:08.332947 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67198dd2-933d-4056-8900-c16dd7c549ea admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:16:08.343779 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67198dd2-933d-4056-8900-c16dd7c549ea admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:16:08.348452 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372c8d88-16ea-4a88-bf42-d28627eefcca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:16:08.349141 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c8d88-16ea-4a88-bf42-d28627eefcca admin admin] REQUEST_METHOD: `PUT` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:08.349403 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c8d88-16ea-4a88-bf42-d28627eefcca admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:08.349676 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c8d88-16ea-4a88-bf42-d28627eefcca admin admin] PATH_INFO: `/v3/projects/21f10a4791f54b0698663c0529674934/users/068ec27639d04e94a7d9d23d74d5c2d9/roles/9bd0470b544240209a420f640c79c06c` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:08.367258 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372c8d88-16ea-4a88-bf42-d28627eefcca admin admin] RBAC: Authorizing `identity:create_grant(project_id=21f10a4791f54b0698663c0529674934, user_id=068ec27639d04e94a7d9d23d74d5c2d9, role_id=9bd0470b544240209a420f640c79c06c)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:16:08.369646 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372c8d88-16ea-4a88-bf42-d28627eefcca admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:16:08.435693 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1234/2445] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:16:08 2022] PUT /identity/v3/projects/21f10a4791f54b0698663c0529674934/users/068ec27639d04e94a7d9d23d74d5c2d9/roles/9bd0470b544240209a420f640c79c06c => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:16:08.447893 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3e6273-7cb7-44e1-9f0f-549bfb22cccf None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:08.448176 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3e6273-7cb7-44e1-9f0f-549bfb22cccf None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:08.448483 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3e6273-7cb7-44e1-9f0f-549bfb22cccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:08.476639 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-ce3e6273-7cb7-44e1-9f0f-549bfb22cccf None None] MFA Rules not processed for user `068ec27639d04e94a7d9d23d74d5c2d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:16:08.477903 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-ce3e6273-7cb7-44e1-9f0f-549bfb22cccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:16:08.542997 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1212/2446] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:16:08 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 03:16:08.547480 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372c8d88-16ea-4a88-bf42-d28627eefcca admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:16:08.610483 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:16:08.614365 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372c8d88-16ea-4a88-bf42-d28627eefcca admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:16:08.619231 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31214040-8b60-4abb-a91f-15c0aa9eca66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:16:08.630693 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31214040-8b60-4abb-a91f-15c0aa9eca66 admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:08.630693 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31214040-8b60-4abb-a91f-15c0aa9eca66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:08.630693 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31214040-8b60-4abb-a91f-15c0aa9eca66 admin admin] PATH_INFO: `/v3/projects` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:08.630693 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31214040-8b60-4abb-a91f-15c0aa9eca66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:16:08.630693 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31214040-8b60-4abb-a91f-15c0aa9eca66 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:16:08.711558 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1235/2447] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:16:08 2022] POST /identity/v3/projects => generated 385 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:16:08.716090 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3e6273-7cb7-44e1-9f0f-549bfb22cccf None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:16:08.726872 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3e6273-7cb7-44e1-9f0f-549bfb22cccf None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:16:08.786307 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cec6f83-2a8a-465a-bd44-233a894f5b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:16:08.786918 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cec6f83-2a8a-465a-bd44-233a894f5b43 admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:08.787201 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cec6f83-2a8a-465a-bd44-233a894f5b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:08.787472 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cec6f83-2a8a-465a-bd44-233a894f5b43 admin admin] PATH_INFO: `/v3/users` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:08.788119 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cec6f83-2a8a-465a-bd44-233a894f5b43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:16:08.790242 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cec6f83-2a8a-465a-bd44-233a894f5b43 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:16:08.799253 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:16:08.931041 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1213/2448] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:16:08 2022] POST /identity/v3/users => generated 299 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:16:08.932670 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31214040-8b60-4abb-a91f-15c0aa9eca66 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:16:08.942900 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31214040-8b60-4abb-a91f-15c0aa9eca66 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:16:08.948057 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bc3701-da07-42a1-9d0a-012e9232564d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:16:08.954558 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc3701-da07-42a1-9d0a-012e9232564d admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:08.954558 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc3701-da07-42a1-9d0a-012e9232564d admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:08.954558 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc3701-da07-42a1-9d0a-012e9232564d admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:08.954558 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bc3701-da07-42a1-9d0a-012e9232564d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:16:08.954558 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bc3701-da07-42a1-9d0a-012e9232564d admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:16:08.961711 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1236/2449] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:16:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:16:08.965864 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cec6f83-2a8a-465a-bd44-233a894f5b43 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:16:08.975965 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cec6f83-2a8a-465a-bd44-233a894f5b43 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:16:08.980828 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e362822a-a509-47b2-a1fd-0184498ca69d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:16:08.981498 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e362822a-a509-47b2-a1fd-0184498ca69d admin admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:08.981799 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e362822a-a509-47b2-a1fd-0184498ca69d admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:08.982077 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e362822a-a509-47b2-a1fd-0184498ca69d admin admin] PATH_INFO: `/v3/projects/e7655d204db248c7aabae545a4495c71/users/882d227bd26345a9b505338072d9335f/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:08.997611 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e362822a-a509-47b2-a1fd-0184498ca69d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7655d204db248c7aabae545a4495c71, user_id=882d227bd26345a9b505338072d9335f, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:16:09.000896 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e362822a-a509-47b2-a1fd-0184498ca69d admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:16:09.085484 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1214/2450] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:16:08 2022] PUT /identity/v3/projects/e7655d204db248c7aabae545a4495c71/users/882d227bd26345a9b505338072d9335f/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 03:16:09.089661 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bc3701-da07-42a1-9d0a-012e9232564d admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:16:09.099831 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bc3701-da07-42a1-9d0a-012e9232564d admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:16:09.156898 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84be7e11-adc1-414d-b681-7e1784ace3a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:16:09.157522 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84be7e11-adc1-414d-b681-7e1784ace3a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:09.157884 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84be7e11-adc1-414d-b681-7e1784ace3a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:09.158165 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84be7e11-adc1-414d-b681-7e1784ace3a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:09.158794 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84be7e11-adc1-414d-b681-7e1784ace3a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:16:09.160387 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84be7e11-adc1-414d-b681-7e1784ace3a8 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:16:09.170534 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1237/2451] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:16:09 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:16:09.175041 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e362822a-a509-47b2-a1fd-0184498ca69d admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:16:09.186529 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e362822a-a509-47b2-a1fd-0184498ca69d admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:16:09.191217 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2405cad4-832e-4ca0-bf5d-a82dd7373dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:16:09.191924 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2405cad4-832e-4ca0-bf5d-a82dd7373dc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:09.192208 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2405cad4-832e-4ca0-bf5d-a82dd7373dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:09.192478 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2405cad4-832e-4ca0-bf5d-a82dd7373dc4 admin admin] PATH_INFO: `/v3/projects/e7655d204db248c7aabae545a4495c71/users/882d227bd26345a9b505338072d9335f/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:09.196447 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2405cad4-832e-4ca0-bf5d-a82dd7373dc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7655d204db248c7aabae545a4495c71, user_id=882d227bd26345a9b505338072d9335f, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:16:09.199204 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2405cad4-832e-4ca0-bf5d-a82dd7373dc4 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:16:09.209966 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1215/2452] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:16:09 2022] PUT /identity/v3/projects/e7655d204db248c7aabae545a4495c71/users/882d227bd26345a9b505338072d9335f/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 03:16:09.220508 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d07f14-0489-40f3-8f8b-c0c9540bbd23 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:09.220803 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d07f14-0489-40f3-8f8b-c0c9540bbd23 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:09.221085 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d07f14-0489-40f3-8f8b-c0c9540bbd23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:09.250681 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-96d07f14-0489-40f3-8f8b-c0c9540bbd23 None None] MFA Rules not processed for user `882d227bd26345a9b505338072d9335f`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:16:09.251908 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-96d07f14-0489-40f3-8f8b-c0c9540bbd23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:16:09.319188 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1238/2453] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:16:09 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:16:09.330013 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2405cad4-832e-4ca0-bf5d-a82dd7373dc4 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:16:09.331694 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-2405cad4-832e-4ca0-bf5d-a82dd7373dc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:16:09.388236 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2405cad4-832e-4ca0-bf5d-a82dd7373dc4 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:16:09.391980 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe23cb7b-64cd-4efa-8aba-43e701fca021 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab599d98b0624a2e89090312bcab713f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:16:09.392579 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe23cb7b-64cd-4efa-8aba-43e701fca021 service cinder] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:09.392860 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe23cb7b-64cd-4efa-8aba-43e701fca021 service cinder] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:09.393133 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe23cb7b-64cd-4efa-8aba-43e701fca021 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:09.451430 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe23cb7b-64cd-4efa-8aba-43e701fca021 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:16:09.452809 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe23cb7b-64cd-4efa-8aba-43e701fca021 service cinder] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:16:09.462268 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1216/2454] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:16:09 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:16:12.927720 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d07f14-0489-40f3-8f8b-c0c9540bbd23 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:16:12.929188 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-96d07f14-0489-40f3-8f8b-c0c9540bbd23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:16:12.988493 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d07f14-0489-40f3-8f8b-c0c9540bbd23 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:16:12.992017 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43df70b9-ab04-4459-a224-24b3d4e60b34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:16:12.992649 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43df70b9-ab04-4459-a224-24b3d4e60b34 service neutron] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:12.992937 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43df70b9-ab04-4459-a224-24b3d4e60b34 service neutron] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:12.993212 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43df70b9-ab04-4459-a224-24b3d4e60b34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:12.995029 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-43df70b9-ab04-4459-a224-24b3d4e60b34 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:16:13.002273 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:16:13.005654 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43df70b9-ab04-4459-a224-24b3d4e60b34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:16:13.006334 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43df70b9-ab04-4459-a224-24b3d4e60b34 service neutron] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:16:13.015550 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1239/2455] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:16:12 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:16:14.513117 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe23cb7b-64cd-4efa-8aba-43e701fca021 service cinder] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:16:14.522413 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe23cb7b-64cd-4efa-8aba-43e701fca021 service cinder] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:16:14.526144 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0b362d-e506-4e9b-9e65-d449469efd65 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab599d98b0624a2e89090312bcab713f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:16:14.526814 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0b362d-e506-4e9b-9e65-d449469efd65 service cinder] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:14.527175 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0b362d-e506-4e9b-9e65-d449469efd65 service cinder] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:14.527517 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0b362d-e506-4e9b-9e65-d449469efd65 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:14.537077 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0b362d-e506-4e9b-9e65-d449469efd65 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:16:14.538613 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0b362d-e506-4e9b-9e65-d449469efd65 service cinder] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:16:14.549619 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1217/2456] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:16:14 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:16:23.318478 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=2 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:16:24.526421 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43df70b9-ab04-4459-a224-24b3d4e60b34 service neutron] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:16:24.544701 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:INSERT=4,keystone:SELECT=71 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:16:24.595905 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43df70b9-ab04-4459-a224-24b3d4e60b34 service neutron] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:16:24.600599 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ab3d10-7c13-4706-8b6c-5a6903e383e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:16:24.601256 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ab3d10-7c13-4706-8b6c-5a6903e383e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:24.601540 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ab3d10-7c13-4706-8b6c-5a6903e383e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:24.601843 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ab3d10-7c13-4706-8b6c-5a6903e383e4 admin admin] PATH_INFO: `/v3/users/068ec27639d04e94a7d9d23d74d5c2d9` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:24.603012 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ab3d10-7c13-4706-8b6c-5a6903e383e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=068ec27639d04e94a7d9d23d74d5c2d9)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:16:24.604966 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ab3d10-7c13-4706-8b6c-5a6903e383e4 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:16:24.840394 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-26ab3d10-7c13-4706-8b6c-5a6903e383e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '068ec27639d04e94a7d9d23d74d5c2d9'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:24.841049 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-26ab3d10-7c13-4706-8b6c-5a6903e383e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '068ec27639d04e94a7d9d23d74d5c2d9'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:24.906721 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-26ab3d10-7c13-4706-8b6c-5a6903e383e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '068ec27639d04e94a7d9d23d74d5c2d9'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:24.907304 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-26ab3d10-7c13-4706-8b6c-5a6903e383e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '068ec27639d04e94a7d9d23d74d5c2d9'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:24.914942 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-26ab3d10-7c13-4706-8b6c-5a6903e383e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '068ec27639d04e94a7d9d23d74d5c2d9'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:24.933460 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1240/2457] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:16:24 2022] DELETE /identity/v3/users/068ec27639d04e94a7d9d23d74d5c2d9 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 03:16:24.938182 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d4761c-b92a-479b-a404-d51326007e88 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:24.938464 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d4761c-b92a-479b-a404-d51326007e88 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:24.938763 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d4761c-b92a-479b-a404-d51326007e88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:24.957828 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-11d4761c-b92a-479b-a404-d51326007e88 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:16:24.959020 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-11d4761c-b92a-479b-a404-d51326007e88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:16:25.018740 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1218/2458] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:16:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:16:25.028537 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ab3d10-7c13-4706-8b6c-5a6903e383e4 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:16:25.033657 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-26ab3d10-7c13-4706-8b6c-5a6903e383e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:16:25.086440 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ab3d10-7c13-4706-8b6c-5a6903e383e4 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:16:25.089981 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1811f008-151b-4b2d-a14b-0d7c7d1379f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:16:25.090582 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1811f008-151b-4b2d-a14b-0d7c7d1379f1 service neutron] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:25.090900 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1811f008-151b-4b2d-a14b-0d7c7d1379f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:25.091143 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1811f008-151b-4b2d-a14b-0d7c7d1379f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:25.100049 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1811f008-151b-4b2d-a14b-0d7c7d1379f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:16:25.101475 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1811f008-151b-4b2d-a14b-0d7c7d1379f1 service neutron] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:16:25.111474 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1241/2459] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:16:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:16:25.545274 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d4761c-b92a-479b-a404-d51326007e88 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:16:25.546710 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-11d4761c-b92a-479b-a404-d51326007e88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:16:25.559947 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d4761c-b92a-479b-a404-d51326007e88 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:16:25.564644 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e188bc9-ac8a-42fb-9bb9-22a9615c130c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:16:25.565374 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e188bc9-ac8a-42fb-9bb9-22a9615c130c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:25.565654 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e188bc9-ac8a-42fb-9bb9-22a9615c130c admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:25.565955 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e188bc9-ac8a-42fb-9bb9-22a9615c130c admin admin] PATH_INFO: `/v3/projects/21f10a4791f54b0698663c0529674934` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:25.567109 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e188bc9-ac8a-42fb-9bb9-22a9615c130c admin admin] RBAC: Authorizing `identity:delete_project(project_id=21f10a4791f54b0698663c0529674934)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:16:25.569061 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e188bc9-ac8a-42fb-9bb9-22a9615c130c admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:16:25.673795 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-6e188bc9-ac8a-42fb-9bb9-22a9615c130c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21f10a4791f54b0698663c0529674934', 'request_id': 'req-6e188bc9-ac8a-42fb-9bb9-22a9615c130c'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:25.680718 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-6e188bc9-ac8a-42fb-9bb9-22a9615c130c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21f10a4791f54b0698663c0529674934', 'request_id': 'req-6e188bc9-ac8a-42fb-9bb9-22a9615c130c'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:25.752240 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-6e188bc9-ac8a-42fb-9bb9-22a9615c130c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21f10a4791f54b0698663c0529674934', 'request_id': 'req-6e188bc9-ac8a-42fb-9bb9-22a9615c130c'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:25.766215 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-6e188bc9-ac8a-42fb-9bb9-22a9615c130c admin admin] The token cache is being invalidate because project 21f10a4791f54b0698663c0529674934 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:16:25.766502 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-6e188bc9-ac8a-42fb-9bb9-22a9615c130c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:25.767150 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-6e188bc9-ac8a-42fb-9bb9-22a9615c130c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:25.768793 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1219/2460] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:16:25 2022] DELETE /identity/v3/projects/21f10a4791f54b0698663c0529674934 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:16:25.773017 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1811f008-151b-4b2d-a14b-0d7c7d1379f1 service neutron] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:16:25.783560 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1811f008-151b-4b2d-a14b-0d7c7d1379f1 service neutron] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:16:25.801566 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:16:25.840389 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76f9bda-a537-49ea-bb64-fe6a13673613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:16:25.841222 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76f9bda-a537-49ea-bb64-fe6a13673613 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:25.841511 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76f9bda-a537-49ea-bb64-fe6a13673613 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:25.841821 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76f9bda-a537-49ea-bb64-fe6a13673613 admin admin] PATH_INFO: `/v3/users/882d227bd26345a9b505338072d9335f` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:25.842940 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76f9bda-a537-49ea-bb64-fe6a13673613 admin admin] RBAC: Authorizing `identity:delete_user(user_id=882d227bd26345a9b505338072d9335f)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:16:25.844905 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76f9bda-a537-49ea-bb64-fe6a13673613 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:16:26.015431 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-e76f9bda-a537-49ea-bb64-fe6a13673613 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '882d227bd26345a9b505338072d9335f'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:26.015951 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-e76f9bda-a537-49ea-bb64-fe6a13673613 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '882d227bd26345a9b505338072d9335f'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:26.097708 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-e76f9bda-a537-49ea-bb64-fe6a13673613 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '882d227bd26345a9b505338072d9335f'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:26.098687 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-e76f9bda-a537-49ea-bb64-fe6a13673613 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '882d227bd26345a9b505338072d9335f'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:26.106210 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-e76f9bda-a537-49ea-bb64-fe6a13673613 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '882d227bd26345a9b505338072d9335f'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:26.124636 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1242/2461] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:16:25 2022] DELETE /identity/v3/users/882d227bd26345a9b505338072d9335f => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 03:16:26.598088 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e188bc9-ac8a-42fb-9bb9-22a9615c130c admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:16:26.599568 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-6e188bc9-ac8a-42fb-9bb9-22a9615c130c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:16:26.613380 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e188bc9-ac8a-42fb-9bb9-22a9615c130c admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:16:26.683354 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94afd65e-df76-49a6-9f65-a3b4ab8a7a44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:16:26.683354 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94afd65e-df76-49a6-9f65-a3b4ab8a7a44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:26.683709 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94afd65e-df76-49a6-9f65-a3b4ab8a7a44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:26.683920 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94afd65e-df76-49a6-9f65-a3b4ab8a7a44 admin admin] PATH_INFO: `/v3/projects/e7655d204db248c7aabae545a4495c71` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:26.685268 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94afd65e-df76-49a6-9f65-a3b4ab8a7a44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7655d204db248c7aabae545a4495c71)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:16:26.687247 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94afd65e-df76-49a6-9f65-a3b4ab8a7a44 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:16:26.771150 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:16:26.787431 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-94afd65e-df76-49a6-9f65-a3b4ab8a7a44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7655d204db248c7aabae545a4495c71', 'request_id': 'req-94afd65e-df76-49a6-9f65-a3b4ab8a7a44'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:26.793565 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-94afd65e-df76-49a6-9f65-a3b4ab8a7a44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7655d204db248c7aabae545a4495c71', 'request_id': 'req-94afd65e-df76-49a6-9f65-a3b4ab8a7a44'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:26.849732 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-94afd65e-df76-49a6-9f65-a3b4ab8a7a44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7655d204db248c7aabae545a4495c71', 'request_id': 'req-94afd65e-df76-49a6-9f65-a3b4ab8a7a44'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:26.860314 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-94afd65e-df76-49a6-9f65-a3b4ab8a7a44 admin admin] The token cache is being invalidate because project e7655d204db248c7aabae545a4495c71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:16:26.860593 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-94afd65e-df76-49a6-9f65-a3b4ab8a7a44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:26.861205 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-94afd65e-df76-49a6-9f65-a3b4ab8a7a44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:26.862960 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1220/2462] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:16:26 2022] DELETE /identity/v3/projects/e7655d204db248c7aabae545a4495c71 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:16:26.868846 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7eee4d-d2e0-46bc-a54f-9804a27aea61 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:26.869345 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7eee4d-d2e0-46bc-a54f-9804a27aea61 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:26.869447 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7eee4d-d2e0-46bc-a54f-9804a27aea61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:26.891289 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-9e7eee4d-d2e0-46bc-a54f-9804a27aea61 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:16:26.892532 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-9e7eee4d-d2e0-46bc-a54f-9804a27aea61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:16:26.950763 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1243/2463] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:16:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:16:26.960188 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c788fe8d-9891-4525-83d7-fd6a4fd7ce4d None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:26.960471 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c788fe8d-9891-4525-83d7-fd6a4fd7ce4d None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:26.960740 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c788fe8d-9891-4525-83d7-fd6a4fd7ce4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:26.980503 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-c788fe8d-9891-4525-83d7-fd6a4fd7ce4d None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:16:26.981732 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-c788fe8d-9891-4525-83d7-fd6a4fd7ce4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:16:26.989846 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1221/2464] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:16:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:16:26.994048 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7eee4d-d2e0-46bc-a54f-9804a27aea61 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:16:27.004272 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7eee4d-d2e0-46bc-a54f-9804a27aea61 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:16:27.008849 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9b1016-8951-4448-ae41-7ed6b4e2f935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:16:27.009846 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9b1016-8951-4448-ae41-7ed6b4e2f935 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:27.009846 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9b1016-8951-4448-ae41-7ed6b4e2f935 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:27.010065 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9b1016-8951-4448-ae41-7ed6b4e2f935 admin admin] PATH_INFO: `/v3/domains` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:27.010647 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9b1016-8951-4448-ae41-7ed6b4e2f935 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 03:16:27.011021 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9b1016-8951-4448-ae41-7ed6b4e2f935 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:16:27.012615 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9b1016-8951-4448-ae41-7ed6b4e2f935 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:16:27.024682 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1244/2465] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Oct 14 03:16:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 03:16:27.029061 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c788fe8d-9891-4525-83d7-fd6a4fd7ce4d None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:16:27.038136 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c788fe8d-9891-4525-83d7-fd6a4fd7ce4d None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:16:27.043270 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb35964-91ed-46e1-bf9e-9e6a381825cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:16:27.046691 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb35964-91ed-46e1-bf9e-9e6a381825cf admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:27.046691 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb35964-91ed-46e1-bf9e-9e6a381825cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:27.046691 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb35964-91ed-46e1-bf9e-9e6a381825cf admin admin] PATH_INFO: `/v3/projects` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:27.046691 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb35964-91ed-46e1-bf9e-9e6a381825cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:16:27.047043 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb35964-91ed-46e1-bf9e-9e6a381825cf admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:16:27.116219 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1222/2466] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:16:27 2022] POST /identity/v3/projects => generated 373 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:16:27.121019 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9b1016-8951-4448-ae41-7ed6b4e2f935 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:16:27.131222 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9b1016-8951-4448-ae41-7ed6b4e2f935 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:16:27.156833 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:16:27.188125 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f34c45d-693a-4c31-9cf9-b99915867e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:16:27.188718 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f34c45d-693a-4c31-9cf9-b99915867e8e admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:27.188992 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f34c45d-693a-4c31-9cf9-b99915867e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:27.189263 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f34c45d-693a-4c31-9cf9-b99915867e8e admin admin] PATH_INFO: `/v3/users` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:27.189955 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f34c45d-693a-4c31-9cf9-b99915867e8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:16:27.191876 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f34c45d-693a-4c31-9cf9-b99915867e8e admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:16:27.284006 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1245/2467] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:16:27 2022] POST /identity/v3/users => generated 287 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:16:27.288025 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb35964-91ed-46e1-bf9e-9e6a381825cf admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:16:27.298864 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb35964-91ed-46e1-bf9e-9e6a381825cf admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:16:27.303795 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe8ade0-b9af-4afc-a731-c04c43499416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:16:27.304392 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe8ade0-b9af-4afc-a731-c04c43499416 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:27.304674 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe8ade0-b9af-4afc-a731-c04c43499416 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:27.304948 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe8ade0-b9af-4afc-a731-c04c43499416 admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:27.305597 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe8ade0-b9af-4afc-a731-c04c43499416 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:16:27.307239 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe8ade0-b9af-4afc-a731-c04c43499416 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:16:27.317906 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1223/2468] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:16:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:16:27.321838 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f34c45d-693a-4c31-9cf9-b99915867e8e admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:16:27.331694 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f34c45d-693a-4c31-9cf9-b99915867e8e admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:16:27.336324 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17266a44-6063-429c-82a0-1d289d5c8042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:16:27.336917 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17266a44-6063-429c-82a0-1d289d5c8042 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:27.337198 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17266a44-6063-429c-82a0-1d289d5c8042 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:27.337469 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17266a44-6063-429c-82a0-1d289d5c8042 admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:27.338143 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17266a44-6063-429c-82a0-1d289d5c8042 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:16:27.339735 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17266a44-6063-429c-82a0-1d289d5c8042 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:16:27.349396 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1246/2469] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:16:27 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:16:27.353279 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe8ade0-b9af-4afc-a731-c04c43499416 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:16:27.363905 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe8ade0-b9af-4afc-a731-c04c43499416 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:16:27.368503 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1cf6ec-f575-4417-a616-eb5ed097e025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:16:27.369142 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1cf6ec-f575-4417-a616-eb5ed097e025 admin admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:27.369425 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1cf6ec-f575-4417-a616-eb5ed097e025 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:27.369700 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1cf6ec-f575-4417-a616-eb5ed097e025 admin admin] PATH_INFO: `/v3/projects/3d1fc9a5992648cf81c8ceccf6964c67/users/228682a3094141f5b4e0f9ac563c63f2/roles/9bd0470b544240209a420f640c79c06c` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:27.385305 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1cf6ec-f575-4417-a616-eb5ed097e025 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d1fc9a5992648cf81c8ceccf6964c67, user_id=228682a3094141f5b4e0f9ac563c63f2, role_id=9bd0470b544240209a420f640c79c06c)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:16:27.387979 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1cf6ec-f575-4417-a616-eb5ed097e025 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:16:27.460991 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1224/2470] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:16:27 2022] PUT /identity/v3/projects/3d1fc9a5992648cf81c8ceccf6964c67/users/228682a3094141f5b4e0f9ac563c63f2/roles/9bd0470b544240209a420f640c79c06c => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:16:27.472888 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0378d3-05ce-4608-9e90-26f0cdca6f25 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:27.473195 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0378d3-05ce-4608-9e90-26f0cdca6f25 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:27.473518 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0378d3-05ce-4608-9e90-26f0cdca6f25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:27.501298 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-9a0378d3-05ce-4608-9e90-26f0cdca6f25 None None] MFA Rules not processed for user `228682a3094141f5b4e0f9ac563c63f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:16:27.502535 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-9a0378d3-05ce-4608-9e90-26f0cdca6f25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:16:27.565542 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1247/2471] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:16:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:16:27.577066 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1cf6ec-f575-4417-a616-eb5ed097e025 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:16:27.578814 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-0c1cf6ec-f575-4417-a616-eb5ed097e025 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:16:27.635414 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1cf6ec-f575-4417-a616-eb5ed097e025 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:16:27.639222 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33324409-19dd-407b-96b1-d61ecef379fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab599d98b0624a2e89090312bcab713f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:16:27.639821 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33324409-19dd-407b-96b1-d61ecef379fe service cinder] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:27.640103 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33324409-19dd-407b-96b1-d61ecef379fe service cinder] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:27.640378 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33324409-19dd-407b-96b1-d61ecef379fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:27.648633 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33324409-19dd-407b-96b1-d61ecef379fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:16:27.650095 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33324409-19dd-407b-96b1-d61ecef379fe service cinder] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:16:27.659248 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1225/2472] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:16:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:16:27.680895 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0378d3-05ce-4608-9e90-26f0cdca6f25 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:16:27.743618 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0378d3-05ce-4608-9e90-26f0cdca6f25 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:16:27.748303 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50185e74-c9fe-4dbe-b8d4-1718fd3012f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:16:27.748940 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50185e74-c9fe-4dbe-b8d4-1718fd3012f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:27.749225 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50185e74-c9fe-4dbe-b8d4-1718fd3012f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:27.749503 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50185e74-c9fe-4dbe-b8d4-1718fd3012f7 admin admin] PATH_INFO: `/v3/users/228682a3094141f5b4e0f9ac563c63f2` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:27.750715 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50185e74-c9fe-4dbe-b8d4-1718fd3012f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=228682a3094141f5b4e0f9ac563c63f2)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:16:27.752663 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50185e74-c9fe-4dbe-b8d4-1718fd3012f7 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:16:27.974223 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:16:27.980058 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-50185e74-c9fe-4dbe-b8d4-1718fd3012f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '228682a3094141f5b4e0f9ac563c63f2'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:27.980722 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-50185e74-c9fe-4dbe-b8d4-1718fd3012f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '228682a3094141f5b4e0f9ac563c63f2'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:28.044006 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-50185e74-c9fe-4dbe-b8d4-1718fd3012f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '228682a3094141f5b4e0f9ac563c63f2'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:28.044641 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-50185e74-c9fe-4dbe-b8d4-1718fd3012f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '228682a3094141f5b4e0f9ac563c63f2'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:28.052378 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-50185e74-c9fe-4dbe-b8d4-1718fd3012f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '228682a3094141f5b4e0f9ac563c63f2'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:28.070069 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1248/2473] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:16:27 2022] DELETE /identity/v3/users/228682a3094141f5b4e0f9ac563c63f2 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:16:28.075105 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0091415-f1d9-4b3f-b0db-f2db1ad2a671 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:28.075389 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0091415-f1d9-4b3f-b0db-f2db1ad2a671 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:28.075665 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0091415-f1d9-4b3f-b0db-f2db1ad2a671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:28.094808 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-e0091415-f1d9-4b3f-b0db-f2db1ad2a671 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:16:28.095964 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-e0091415-f1d9-4b3f-b0db-f2db1ad2a671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:16:28.142682 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:16:28.154295 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1226/2474] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:16:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:16:28.164096 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50185e74-c9fe-4dbe-b8d4-1718fd3012f7 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:16:28.165519 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-50185e74-c9fe-4dbe-b8d4-1718fd3012f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:16:28.223327 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50185e74-c9fe-4dbe-b8d4-1718fd3012f7 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:16:28.227031 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c29225a-f353-45c5-b671-43ee7af16762 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:16:28.227633 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c29225a-f353-45c5-b671-43ee7af16762 service neutron] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:28.227915 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c29225a-f353-45c5-b671-43ee7af16762 service neutron] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:28.228191 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c29225a-f353-45c5-b671-43ee7af16762 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:28.237356 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c29225a-f353-45c5-b671-43ee7af16762 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:16:28.239114 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c29225a-f353-45c5-b671-43ee7af16762 service neutron] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:16:28.249134 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1249/2475] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:16:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:16:28.662359 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0091415-f1d9-4b3f-b0db-f2db1ad2a671 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:16:28.663822 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-e0091415-f1d9-4b3f-b0db-f2db1ad2a671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:16:28.678084 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0091415-f1d9-4b3f-b0db-f2db1ad2a671 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:16:28.683410 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d025ad-54bc-430e-b6fe-bd2fb474d8a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:16:28.684092 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d025ad-54bc-430e-b6fe-bd2fb474d8a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:28.684377 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d025ad-54bc-430e-b6fe-bd2fb474d8a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:28.684720 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d025ad-54bc-430e-b6fe-bd2fb474d8a6 admin admin] PATH_INFO: `/v3/projects/3d1fc9a5992648cf81c8ceccf6964c67` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:28.685800 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d025ad-54bc-430e-b6fe-bd2fb474d8a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d1fc9a5992648cf81c8ceccf6964c67)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:16:28.687814 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d025ad-54bc-430e-b6fe-bd2fb474d8a6 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:16:28.792665 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-f1d025ad-54bc-430e-b6fe-bd2fb474d8a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d1fc9a5992648cf81c8ceccf6964c67', 'request_id': 'req-f1d025ad-54bc-430e-b6fe-bd2fb474d8a6'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:28.798781 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-f1d025ad-54bc-430e-b6fe-bd2fb474d8a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d1fc9a5992648cf81c8ceccf6964c67', 'request_id': 'req-f1d025ad-54bc-430e-b6fe-bd2fb474d8a6'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:28.878178 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-f1d025ad-54bc-430e-b6fe-bd2fb474d8a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d1fc9a5992648cf81c8ceccf6964c67', 'request_id': 'req-f1d025ad-54bc-430e-b6fe-bd2fb474d8a6'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:28.889360 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-f1d025ad-54bc-430e-b6fe-bd2fb474d8a6 admin admin] The token cache is being invalidate because project 3d1fc9a5992648cf81c8ceccf6964c67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:16:28.889645 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-f1d025ad-54bc-430e-b6fe-bd2fb474d8a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:28.890343 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-f1d025ad-54bc-430e-b6fe-bd2fb474d8a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:28.891828 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1227/2476] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:16:28 2022] DELETE /identity/v3/projects/3d1fc9a5992648cf81c8ceccf6964c67 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:16:28.896679 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9137113-42e5-4949-a0a5-569b01642bd4 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:28.897178 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9137113-42e5-4949-a0a5-569b01642bd4 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:28.897473 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9137113-42e5-4949-a0a5-569b01642bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:28.918645 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-b9137113-42e5-4949-a0a5-569b01642bd4 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:16:28.919771 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-b9137113-42e5-4949-a0a5-569b01642bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:16:28.978719 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1250/2477] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:16:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:16:28.986789 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93071441-8c4a-49df-a543-ff547c66ff0b None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:28.987073 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93071441-8c4a-49df-a543-ff547c66ff0b None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:28.987348 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93071441-8c4a-49df-a543-ff547c66ff0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:29.007544 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-93071441-8c4a-49df-a543-ff547c66ff0b None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:16:29.008757 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-93071441-8c4a-49df-a543-ff547c66ff0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:16:29.017131 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1228/2478] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:16:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:16:29.026097 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9137113-42e5-4949-a0a5-569b01642bd4 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:16:29.036225 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9137113-42e5-4949-a0a5-569b01642bd4 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:16:29.040837 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06838cc-8973-4af0-9a46-4ec5d487f2ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:16:29.041423 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06838cc-8973-4af0-9a46-4ec5d487f2ff admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:29.041701 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06838cc-8973-4af0-9a46-4ec5d487f2ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:29.042029 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06838cc-8973-4af0-9a46-4ec5d487f2ff admin admin] PATH_INFO: `/v3/domains` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:29.043251 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06838cc-8973-4af0-9a46-4ec5d487f2ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 03:16:29.043555 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06838cc-8973-4af0-9a46-4ec5d487f2ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:16:29.045145 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06838cc-8973-4af0-9a46-4ec5d487f2ff admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:16:29.056328 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1251/2479] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Oct 14 03:16:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 03:16:29.060450 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93071441-8c4a-49df-a543-ff547c66ff0b None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:16:29.069634 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93071441-8c4a-49df-a543-ff547c66ff0b None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:16:29.074308 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e153556-e9df-4e2c-9970-a7fbfd76b054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:16:29.074930 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e153556-e9df-4e2c-9970-a7fbfd76b054 admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:29.075208 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e153556-e9df-4e2c-9970-a7fbfd76b054 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:29.075480 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e153556-e9df-4e2c-9970-a7fbfd76b054 admin admin] PATH_INFO: `/v3/projects` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:29.076116 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e153556-e9df-4e2c-9970-a7fbfd76b054 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:16:29.078026 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e153556-e9df-4e2c-9970-a7fbfd76b054 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:16:29.196773 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1229/2480] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:16:29 2022] POST /identity/v3/projects => generated 375 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:16:29.200180 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06838cc-8973-4af0-9a46-4ec5d487f2ff admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:16:29.210114 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06838cc-8973-4af0-9a46-4ec5d487f2ff admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:16:29.253453 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:16:29.266331 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457bb4fc-7d45-4193-9158-a53af6d1632e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:16:29.266921 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457bb4fc-7d45-4193-9158-a53af6d1632e admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:29.267202 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457bb4fc-7d45-4193-9158-a53af6d1632e admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:29.267509 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457bb4fc-7d45-4193-9158-a53af6d1632e admin admin] PATH_INFO: `/v3/users` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:29.268138 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457bb4fc-7d45-4193-9158-a53af6d1632e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:16:29.270079 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457bb4fc-7d45-4193-9158-a53af6d1632e admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:16:29.384108 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1252/2481] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:16:29 2022] POST /identity/v3/users => generated 288 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:16:29.391734 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e153556-e9df-4e2c-9970-a7fbfd76b054 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:16:29.403250 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e153556-e9df-4e2c-9970-a7fbfd76b054 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:16:29.407968 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f14e949-843e-4f19-9809-c4a6f6ae1cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:16:29.408562 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f14e949-843e-4f19-9809-c4a6f6ae1cf2 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:29.408843 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f14e949-843e-4f19-9809-c4a6f6ae1cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:29.409117 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f14e949-843e-4f19-9809-c4a6f6ae1cf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:29.409775 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f14e949-843e-4f19-9809-c4a6f6ae1cf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:16:29.411377 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f14e949-843e-4f19-9809-c4a6f6ae1cf2 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:16:29.421191 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1230/2482] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:16:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:16:29.425124 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457bb4fc-7d45-4193-9158-a53af6d1632e admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:16:29.434836 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457bb4fc-7d45-4193-9158-a53af6d1632e admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:16:29.439414 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2505de98-792b-4615-9db3-976769135af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:16:29.440000 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2505de98-792b-4615-9db3-976769135af2 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:29.440289 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2505de98-792b-4615-9db3-976769135af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:29.440568 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2505de98-792b-4615-9db3-976769135af2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:29.441202 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2505de98-792b-4615-9db3-976769135af2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:16:29.442820 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2505de98-792b-4615-9db3-976769135af2 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:16:29.454904 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1253/2483] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:16:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:16:29.458820 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f14e949-843e-4f19-9809-c4a6f6ae1cf2 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:16:29.469268 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f14e949-843e-4f19-9809-c4a6f6ae1cf2 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:16:29.473937 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586c152d-01a9-4771-be8f-a1ae104cdad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:16:29.474575 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586c152d-01a9-4771-be8f-a1ae104cdad1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:29.474860 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586c152d-01a9-4771-be8f-a1ae104cdad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:29.475138 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586c152d-01a9-4771-be8f-a1ae104cdad1 admin admin] PATH_INFO: `/v3/projects/2e2e309242af4ebf80efeacdc30d8152/users/e7b3003e5d1f42b699e497ae54f934b5/roles/9bd0470b544240209a420f640c79c06c` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:29.489627 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586c152d-01a9-4771-be8f-a1ae104cdad1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e2e309242af4ebf80efeacdc30d8152, user_id=e7b3003e5d1f42b699e497ae54f934b5, role_id=9bd0470b544240209a420f640c79c06c)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:16:29.492313 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586c152d-01a9-4771-be8f-a1ae104cdad1 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:16:29.578238 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1231/2484] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:16:29 2022] PUT /identity/v3/projects/2e2e309242af4ebf80efeacdc30d8152/users/e7b3003e5d1f42b699e497ae54f934b5/roles/9bd0470b544240209a420f640c79c06c => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 03:16:29.589983 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a625e2-c13d-4900-8327-482bce65cb0b None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:29.590298 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a625e2-c13d-4900-8327-482bce65cb0b None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:29.590570 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a625e2-c13d-4900-8327-482bce65cb0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:29.617841 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-49a625e2-c13d-4900-8327-482bce65cb0b None None] MFA Rules not processed for user `e7b3003e5d1f42b699e497ae54f934b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:16:29.619046 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-49a625e2-c13d-4900-8327-482bce65cb0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:16:29.683334 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1254/2485] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:16:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:16:29.694431 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586c152d-01a9-4771-be8f-a1ae104cdad1 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:16:29.696090 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-586c152d-01a9-4771-be8f-a1ae104cdad1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:16:29.753698 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586c152d-01a9-4771-be8f-a1ae104cdad1 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:16:29.757303 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e1b336-4626-4cf4-91d6-68057416304c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a3cd1fbe6df48f8aa90d4ea69905e9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:16:29.757939 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e1b336-4626-4cf4-91d6-68057416304c service glance] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:29.760534 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e1b336-4626-4cf4-91d6-68057416304c service glance] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:29.760534 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e1b336-4626-4cf4-91d6-68057416304c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:29.767391 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e1b336-4626-4cf4-91d6-68057416304c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:16:29.768807 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e1b336-4626-4cf4-91d6-68057416304c service glance] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:16:29.777636 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1232/2486] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:16:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:16:35.604414 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a625e2-c13d-4900-8327-482bce65cb0b None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:16:35.606497 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-49a625e2-c13d-4900-8327-482bce65cb0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:16:35.673581 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a625e2-c13d-4900-8327-482bce65cb0b None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:16:35.678157 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d1c4d6-d4bc-47d7-8ccf-379cc926c356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:16:35.678791 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d1c4d6-d4bc-47d7-8ccf-379cc926c356 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:35.679075 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d1c4d6-d4bc-47d7-8ccf-379cc926c356 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:35.679353 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d1c4d6-d4bc-47d7-8ccf-379cc926c356 admin admin] PATH_INFO: `/v3/users/0f745e320f8c4cd5b1772054dc3c03cf` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:35.680475 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d1c4d6-d4bc-47d7-8ccf-379cc926c356 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f745e320f8c4cd5b1772054dc3c03cf)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:16:35.682448 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d1c4d6-d4bc-47d7-8ccf-379cc926c356 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:16:35.859330 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-74d1c4d6-d4bc-47d7-8ccf-379cc926c356 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f745e320f8c4cd5b1772054dc3c03cf'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:35.859914 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-74d1c4d6-d4bc-47d7-8ccf-379cc926c356 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f745e320f8c4cd5b1772054dc3c03cf'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:35.918905 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-74d1c4d6-d4bc-47d7-8ccf-379cc926c356 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f745e320f8c4cd5b1772054dc3c03cf'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:35.919747 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-74d1c4d6-d4bc-47d7-8ccf-379cc926c356 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f745e320f8c4cd5b1772054dc3c03cf'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:35.922296 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:16:35.933012 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-74d1c4d6-d4bc-47d7-8ccf-379cc926c356 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f745e320f8c4cd5b1772054dc3c03cf'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:35.953880 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1255/2487] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:16:35 2022] DELETE /identity/v3/users/0f745e320f8c4cd5b1772054dc3c03cf => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:16:35.959429 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c84c9b-da71-43dd-92e5-312ab6b9eb29 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:35.959429 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c84c9b-da71-43dd-92e5-312ab6b9eb29 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:35.959429 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c84c9b-da71-43dd-92e5-312ab6b9eb29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:35.975536 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-53c84c9b-da71-43dd-92e5-312ab6b9eb29 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:16:35.976736 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-53c84c9b-da71-43dd-92e5-312ab6b9eb29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:16:35.986672 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:16:36.055718 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1233/2488] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:16:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 03:16:36.065287 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d1c4d6-d4bc-47d7-8ccf-379cc926c356 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:16:36.068248 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-74d1c4d6-d4bc-47d7-8ccf-379cc926c356 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:16:36.144109 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d1c4d6-d4bc-47d7-8ccf-379cc926c356 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:16:36.148126 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d5904b-ce57-48cf-be90-656b842dd7df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:16:36.148822 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d5904b-ce57-48cf-be90-656b842dd7df service neutron] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:36.149174 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d5904b-ce57-48cf-be90-656b842dd7df service neutron] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:36.151220 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d5904b-ce57-48cf-be90-656b842dd7df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:36.162145 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d5904b-ce57-48cf-be90-656b842dd7df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:16:36.164913 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d5904b-ce57-48cf-be90-656b842dd7df service neutron] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:16:36.175444 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1256/2489] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:16:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 03:16:36.287644 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c84c9b-da71-43dd-92e5-312ab6b9eb29 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:16:36.299699 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c84c9b-da71-43dd-92e5-312ab6b9eb29 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:16:36.304170 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e91e020-d895-4227-ae01-67a03cc83f9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:16:36.304802 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e91e020-d895-4227-ae01-67a03cc83f9f service neutron] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:36.305147 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e91e020-d895-4227-ae01-67a03cc83f9f service neutron] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:36.305398 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e91e020-d895-4227-ae01-67a03cc83f9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:36.307313 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-5e91e020-d895-4227-ae01-67a03cc83f9f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:16:36.378741 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e91e020-d895-4227-ae01-67a03cc83f9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:16:36.380325 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e91e020-d895-4227-ae01-67a03cc83f9f service neutron] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:16:36.390983 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1234/2490] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:16:36 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:16:36.559545 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d5904b-ce57-48cf-be90-656b842dd7df service neutron] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:16:36.559545 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-14d5904b-ce57-48cf-be90-656b842dd7df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:16:36.571840 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d5904b-ce57-48cf-be90-656b842dd7df service neutron] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:16:36.588840 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180a9558-2ef5-45ae-bfc7-d28b3bceb8cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:16:36.589747 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a9558-2ef5-45ae-bfc7-d28b3bceb8cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:36.590271 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a9558-2ef5-45ae-bfc7-d28b3bceb8cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:36.590528 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a9558-2ef5-45ae-bfc7-d28b3bceb8cf admin admin] PATH_INFO: `/v3/projects/b3bd42abc89e4835935cf6cb07a668b9` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:36.592332 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180a9558-2ef5-45ae-bfc7-d28b3bceb8cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3bd42abc89e4835935cf6cb07a668b9)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:16:36.595352 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180a9558-2ef5-45ae-bfc7-d28b3bceb8cf admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:16:36.741339 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-180a9558-2ef5-45ae-bfc7-d28b3bceb8cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3bd42abc89e4835935cf6cb07a668b9', 'request_id': 'req-180a9558-2ef5-45ae-bfc7-d28b3bceb8cf'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:36.748376 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-180a9558-2ef5-45ae-bfc7-d28b3bceb8cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3bd42abc89e4835935cf6cb07a668b9', 'request_id': 'req-180a9558-2ef5-45ae-bfc7-d28b3bceb8cf'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:36.815709 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-180a9558-2ef5-45ae-bfc7-d28b3bceb8cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3bd42abc89e4835935cf6cb07a668b9', 'request_id': 'req-180a9558-2ef5-45ae-bfc7-d28b3bceb8cf'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:36.827390 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-180a9558-2ef5-45ae-bfc7-d28b3bceb8cf admin admin] The token cache is being invalidate because project b3bd42abc89e4835935cf6cb07a668b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81517) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:16:36.827718 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-180a9558-2ef5-45ae-bfc7-d28b3bceb8cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:36.828367 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-180a9558-2ef5-45ae-bfc7-d28b3bceb8cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:36.833571 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1257/2491] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:16:36 2022] DELETE /identity/v3/projects/b3bd42abc89e4835935cf6cb07a668b9 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 03:16:36.835117 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e8434f-e0ce-4805-be39-8049132586d1 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:36.835607 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e8434f-e0ce-4805-be39-8049132586d1 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:36.835954 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e8434f-e0ce-4805-be39-8049132586d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:36.863501 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-07e8434f-e0ce-4805-be39-8049132586d1 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:16:36.865168 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-07e8434f-e0ce-4805-be39-8049132586d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:16:36.923788 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1235/2492] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:16:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:16:36.931568 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c99bbe-4626-4492-b3c7-ca63b03a1f67 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:36.931848 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c99bbe-4626-4492-b3c7-ca63b03a1f67 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:36.932142 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c99bbe-4626-4492-b3c7-ca63b03a1f67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:36.952588 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-e9c99bbe-4626-4492-b3c7-ca63b03a1f67 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:16:36.953940 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-e9c99bbe-4626-4492-b3c7-ca63b03a1f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:16:36.961818 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1258/2493] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:16:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:16:36.966237 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e8434f-e0ce-4805-be39-8049132586d1 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:16:36.976422 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e8434f-e0ce-4805-be39-8049132586d1 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:16:36.983296 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345026c7-42fc-4620-b82f-345440c65db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:16:36.983923 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345026c7-42fc-4620-b82f-345440c65db1 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:36.984524 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345026c7-42fc-4620-b82f-345440c65db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:36.984524 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345026c7-42fc-4620-b82f-345440c65db1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:36.985182 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345026c7-42fc-4620-b82f-345440c65db1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 03:16:36.985495 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345026c7-42fc-4620-b82f-345440c65db1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:16:36.987156 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345026c7-42fc-4620-b82f-345440c65db1 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:16:37.016622 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1236/2494] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Oct 14 03:16:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 03:16:37.020632 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c99bbe-4626-4492-b3c7-ca63b03a1f67 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:16:37.030446 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c99bbe-4626-4492-b3c7-ca63b03a1f67 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:16:37.037354 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d8d498-e71e-49a5-8bff-9f1768090c0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:16:37.039530 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d8d498-e71e-49a5-8bff-9f1768090c0b admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:37.039530 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d8d498-e71e-49a5-8bff-9f1768090c0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:37.039530 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d8d498-e71e-49a5-8bff-9f1768090c0b admin admin] PATH_INFO: `/v3/projects` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:37.039530 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d8d498-e71e-49a5-8bff-9f1768090c0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:16:37.054030 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d8d498-e71e-49a5-8bff-9f1768090c0b admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:16:37.131158 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1259/2495] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:16:37 2022] POST /identity/v3/projects => generated 381 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:16:37.132591 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345026c7-42fc-4620-b82f-345440c65db1 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:16:37.142715 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345026c7-42fc-4620-b82f-345440c65db1 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:16:37.182376 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=100 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:16:37.206883 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bb4082-79bf-4990-a0bb-a7a583565f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:16:37.207498 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bb4082-79bf-4990-a0bb-a7a583565f85 admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:37.207779 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bb4082-79bf-4990-a0bb-a7a583565f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:37.208052 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bb4082-79bf-4990-a0bb-a7a583565f85 admin admin] PATH_INFO: `/v3/users` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:37.208709 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bb4082-79bf-4990-a0bb-a7a583565f85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:16:37.210698 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bb4082-79bf-4990-a0bb-a7a583565f85 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:16:37.330002 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1237/2496] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:16:37 2022] POST /identity/v3/users => generated 291 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:16:37.331300 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d8d498-e71e-49a5-8bff-9f1768090c0b admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:16:37.342430 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d8d498-e71e-49a5-8bff-9f1768090c0b admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:16:37.347391 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499b6672-187a-4857-b8c5-5188f85d497b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:16:37.348073 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499b6672-187a-4857-b8c5-5188f85d497b admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:37.348352 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499b6672-187a-4857-b8c5-5188f85d497b admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:37.348630 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499b6672-187a-4857-b8c5-5188f85d497b admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:37.349277 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499b6672-187a-4857-b8c5-5188f85d497b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:16:37.350934 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499b6672-187a-4857-b8c5-5188f85d497b admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:16:37.361731 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1260/2497] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:16:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:16:37.366181 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bb4082-79bf-4990-a0bb-a7a583565f85 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:16:37.376568 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bb4082-79bf-4990-a0bb-a7a583565f85 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:16:37.381463 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340aa3bf-9076-4274-ab8e-87f46e0c84a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:16:37.382104 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340aa3bf-9076-4274-ab8e-87f46e0c84a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:37.382392 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340aa3bf-9076-4274-ab8e-87f46e0c84a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:37.382666 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340aa3bf-9076-4274-ab8e-87f46e0c84a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:37.383310 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340aa3bf-9076-4274-ab8e-87f46e0c84a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:16:37.384902 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340aa3bf-9076-4274-ab8e-87f46e0c84a9 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:16:37.395889 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1238/2498] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:16:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:16:37.400108 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499b6672-187a-4857-b8c5-5188f85d497b admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:16:37.410710 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499b6672-187a-4857-b8c5-5188f85d497b admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:16:37.415240 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288fb757-6921-4f80-a3bb-9db82d049be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:16:37.415882 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288fb757-6921-4f80-a3bb-9db82d049be2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:37.416162 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288fb757-6921-4f80-a3bb-9db82d049be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:37.416488 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288fb757-6921-4f80-a3bb-9db82d049be2 admin admin] PATH_INFO: `/v3/projects/ae722e93c0df4a8cb1dc69b660774a00/users/5e73141873ee40abb83f95d44cfc9a72/roles/9bd0470b544240209a420f640c79c06c` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:37.434226 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288fb757-6921-4f80-a3bb-9db82d049be2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae722e93c0df4a8cb1dc69b660774a00, user_id=5e73141873ee40abb83f95d44cfc9a72, role_id=9bd0470b544240209a420f640c79c06c)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:16:37.437062 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288fb757-6921-4f80-a3bb-9db82d049be2 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:16:37.440539 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:16:37.531100 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1261/2499] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:16:37 2022] PUT /identity/v3/projects/ae722e93c0df4a8cb1dc69b660774a00/users/5e73141873ee40abb83f95d44cfc9a72/roles/9bd0470b544240209a420f640c79c06c => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:16:37.541474 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4172d0-b408-4a0f-a8e7-9a94c48b2af5 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:37.541787 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4172d0-b408-4a0f-a8e7-9a94c48b2af5 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:37.542065 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4172d0-b408-4a0f-a8e7-9a94c48b2af5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:37.571165 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-0e4172d0-b408-4a0f-a8e7-9a94c48b2af5 None None] MFA Rules not processed for user `5e73141873ee40abb83f95d44cfc9a72`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:16:37.572527 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-0e4172d0-b408-4a0f-a8e7-9a94c48b2af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:16:37.640256 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1239/2500] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:16:37 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:16:37.651650 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288fb757-6921-4f80-a3bb-9db82d049be2 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:16:37.654013 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-288fb757-6921-4f80-a3bb-9db82d049be2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:16:37.717728 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288fb757-6921-4f80-a3bb-9db82d049be2 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:16:37.721368 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6221360d-0c4e-41d9-9452-81a428eec8e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab599d98b0624a2e89090312bcab713f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:16:37.722011 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221360d-0c4e-41d9-9452-81a428eec8e9 service cinder] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:37.722294 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221360d-0c4e-41d9-9452-81a428eec8e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:37.722569 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221360d-0c4e-41d9-9452-81a428eec8e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:37.730879 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6221360d-0c4e-41d9-9452-81a428eec8e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:16:37.734825 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6221360d-0c4e-41d9-9452-81a428eec8e9 service cinder] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:16:37.741238 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1262/2501] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:16:37 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:16:40.738213 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4172d0-b408-4a0f-a8e7-9a94c48b2af5 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:16:40.798897 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4172d0-b408-4a0f-a8e7-9a94c48b2af5 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:16:40.803360 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d8ca98-3a15-4441-9853-2a13008f0a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:16:40.803978 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d8ca98-3a15-4441-9853-2a13008f0a42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:40.804253 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d8ca98-3a15-4441-9853-2a13008f0a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:40.804521 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d8ca98-3a15-4441-9853-2a13008f0a42 admin admin] PATH_INFO: `/v3/users/5e73141873ee40abb83f95d44cfc9a72` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:40.805612 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d8ca98-3a15-4441-9853-2a13008f0a42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e73141873ee40abb83f95d44cfc9a72)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:16:40.807555 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d8ca98-3a15-4441-9853-2a13008f0a42 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:16:40.982508 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-19d8ca98-3a15-4441-9853-2a13008f0a42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e73141873ee40abb83f95d44cfc9a72'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:40.983586 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:16:40.990167 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-19d8ca98-3a15-4441-9853-2a13008f0a42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e73141873ee40abb83f95d44cfc9a72'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:41.002447 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-19d8ca98-3a15-4441-9853-2a13008f0a42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e73141873ee40abb83f95d44cfc9a72'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:41.003127 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-19d8ca98-3a15-4441-9853-2a13008f0a42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e73141873ee40abb83f95d44cfc9a72'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:41.078355 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-19d8ca98-3a15-4441-9853-2a13008f0a42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e73141873ee40abb83f95d44cfc9a72'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:41.084951 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1240/2502] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:16:40 2022] DELETE /identity/v3/users/5e73141873ee40abb83f95d44cfc9a72 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:16:41.089427 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9e35dd-3b40-4d0d-b10a-34f6532c1a13 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:41.089967 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9e35dd-3b40-4d0d-b10a-34f6532c1a13 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:41.090108 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9e35dd-3b40-4d0d-b10a-34f6532c1a13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:41.108643 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-cf9e35dd-3b40-4d0d-b10a-34f6532c1a13 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:16:41.109867 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-cf9e35dd-3b40-4d0d-b10a-34f6532c1a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:16:41.168915 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1263/2503] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:16:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:16:41.178891 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d8ca98-3a15-4441-9853-2a13008f0a42 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:16:41.180346 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-19d8ca98-3a15-4441-9853-2a13008f0a42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:16:41.239867 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d8ca98-3a15-4441-9853-2a13008f0a42 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:16:41.243416 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b249fe74-0ef3-4f8e-b4e1-9f21d1be599a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:16:41.244011 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b249fe74-0ef3-4f8e-b4e1-9f21d1be599a service neutron] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:41.244290 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b249fe74-0ef3-4f8e-b4e1-9f21d1be599a service neutron] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:41.244563 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b249fe74-0ef3-4f8e-b4e1-9f21d1be599a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:41.254166 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b249fe74-0ef3-4f8e-b4e1-9f21d1be599a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:16:41.255582 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b249fe74-0ef3-4f8e-b4e1-9f21d1be599a service neutron] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:16:41.269507 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1241/2504] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:16:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:16:41.669495 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9e35dd-3b40-4d0d-b10a-34f6532c1a13 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:16:41.670954 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-cf9e35dd-3b40-4d0d-b10a-34f6532c1a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:16:41.683080 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9e35dd-3b40-4d0d-b10a-34f6532c1a13 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:16:41.687673 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498ba27a-a6f1-4480-bde5-851b237235ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:16:41.688354 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498ba27a-a6f1-4480-bde5-851b237235ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:41.688637 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498ba27a-a6f1-4480-bde5-851b237235ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:41.688915 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498ba27a-a6f1-4480-bde5-851b237235ae admin admin] PATH_INFO: `/v3/projects/ae722e93c0df4a8cb1dc69b660774a00` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:41.690070 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498ba27a-a6f1-4480-bde5-851b237235ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae722e93c0df4a8cb1dc69b660774a00)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:16:41.691980 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498ba27a-a6f1-4480-bde5-851b237235ae admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:16:41.805389 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-498ba27a-a6f1-4480-bde5-851b237235ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae722e93c0df4a8cb1dc69b660774a00', 'request_id': 'req-498ba27a-a6f1-4480-bde5-851b237235ae'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:41.811708 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-498ba27a-a6f1-4480-bde5-851b237235ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae722e93c0df4a8cb1dc69b660774a00', 'request_id': 'req-498ba27a-a6f1-4480-bde5-851b237235ae'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:41.866729 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-498ba27a-a6f1-4480-bde5-851b237235ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae722e93c0df4a8cb1dc69b660774a00', 'request_id': 'req-498ba27a-a6f1-4480-bde5-851b237235ae'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:41.877513 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-498ba27a-a6f1-4480-bde5-851b237235ae admin admin] The token cache is being invalidate because project ae722e93c0df4a8cb1dc69b660774a00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81517) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:16:41.877835 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-498ba27a-a6f1-4480-bde5-851b237235ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:41.878451 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-498ba27a-a6f1-4480-bde5-851b237235ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:41.879928 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1264/2505] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:16:41 2022] DELETE /identity/v3/projects/ae722e93c0df4a8cb1dc69b660774a00 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:16:41.884903 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003e143a-0aaa-4c88-90f3-927d50dafe33 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:41.885396 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003e143a-0aaa-4c88-90f3-927d50dafe33 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:41.885497 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003e143a-0aaa-4c88-90f3-927d50dafe33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:41.909902 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-003e143a-0aaa-4c88-90f3-927d50dafe33 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:16:41.911148 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-003e143a-0aaa-4c88-90f3-927d50dafe33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:16:41.970632 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1242/2506] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:16:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:16:42.033683 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df149e92-4a77-4831-a879-77b6bfba0df1 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:42.034023 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df149e92-4a77-4831-a879-77b6bfba0df1 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:42.034315 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df149e92-4a77-4831-a879-77b6bfba0df1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:42.055141 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-df149e92-4a77-4831-a879-77b6bfba0df1 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:16:42.056440 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-df149e92-4a77-4831-a879-77b6bfba0df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:16:42.064553 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1265/2507] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:16:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:16:42.068860 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003e143a-0aaa-4c88-90f3-927d50dafe33 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:16:42.079154 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003e143a-0aaa-4c88-90f3-927d50dafe33 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:16:42.083815 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0b456a-2c3c-4bac-8e4f-525b077ec177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:16:42.084415 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b456a-2c3c-4bac-8e4f-525b077ec177 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:42.084753 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b456a-2c3c-4bac-8e4f-525b077ec177 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:42.085033 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b456a-2c3c-4bac-8e4f-525b077ec177 admin admin] PATH_INFO: `/v3/domains` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:42.085710 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0b456a-2c3c-4bac-8e4f-525b077ec177 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 03:16:42.086060 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0b456a-2c3c-4bac-8e4f-525b077ec177 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:16:42.087658 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0b456a-2c3c-4bac-8e4f-525b077ec177 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:16:42.099698 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1243/2508] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Oct 14 03:16:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 03:16:42.103947 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df149e92-4a77-4831-a879-77b6bfba0df1 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:16:42.113228 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df149e92-4a77-4831-a879-77b6bfba0df1 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:16:42.117804 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2982fd5d-cc84-46ea-91d4-9453463dd7bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:16:42.118394 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2982fd5d-cc84-46ea-91d4-9453463dd7bf admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:42.118675 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2982fd5d-cc84-46ea-91d4-9453463dd7bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:42.118948 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2982fd5d-cc84-46ea-91d4-9453463dd7bf admin admin] PATH_INFO: `/v3/projects` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:42.119571 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2982fd5d-cc84-46ea-91d4-9453463dd7bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:16:42.121481 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2982fd5d-cc84-46ea-91d4-9453463dd7bf admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:16:42.136097 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:16:42.213240 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1266/2509] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:16:42 2022] POST /identity/v3/projects => generated 371 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:16:42.217914 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0b456a-2c3c-4bac-8e4f-525b077ec177 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:16:42.227769 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0b456a-2c3c-4bac-8e4f-525b077ec177 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:16:42.277154 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:16:42.284688 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e640058f-dce5-4dfd-ac7b-1ddd95c21788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:16:42.285295 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e640058f-dce5-4dfd-ac7b-1ddd95c21788 admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:42.285578 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e640058f-dce5-4dfd-ac7b-1ddd95c21788 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:42.286174 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e640058f-dce5-4dfd-ac7b-1ddd95c21788 admin admin] PATH_INFO: `/v3/users` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:42.286510 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e640058f-dce5-4dfd-ac7b-1ddd95c21788 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:16:42.288446 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e640058f-dce5-4dfd-ac7b-1ddd95c21788 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:16:42.364317 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1244/2510] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:16:42 2022] POST /identity/v3/users => generated 286 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 03:16:42.368401 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2982fd5d-cc84-46ea-91d4-9453463dd7bf admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:16:42.379554 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2982fd5d-cc84-46ea-91d4-9453463dd7bf admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:16:42.384433 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a53309f-edb3-4595-a564-b802339e15fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:16:42.386320 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a53309f-edb3-4595-a564-b802339e15fa admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:42.386320 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a53309f-edb3-4595-a564-b802339e15fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:42.386320 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a53309f-edb3-4595-a564-b802339e15fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:42.386320 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a53309f-edb3-4595-a564-b802339e15fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:16:42.388538 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a53309f-edb3-4595-a564-b802339e15fa admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:16:42.398456 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1267/2511] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:16:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:16:42.402345 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e640058f-dce5-4dfd-ac7b-1ddd95c21788 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:16:42.412762 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e640058f-dce5-4dfd-ac7b-1ddd95c21788 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:16:42.417357 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527e9c7e-1faa-40c8-8eb9-3e6ee0fe8db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:16:42.417977 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e9c7e-1faa-40c8-8eb9-3e6ee0fe8db4 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:42.418266 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e9c7e-1faa-40c8-8eb9-3e6ee0fe8db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:42.418538 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e9c7e-1faa-40c8-8eb9-3e6ee0fe8db4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:42.419174 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527e9c7e-1faa-40c8-8eb9-3e6ee0fe8db4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:16:42.420766 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527e9c7e-1faa-40c8-8eb9-3e6ee0fe8db4 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:16:42.432445 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1245/2512] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:16:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 03:16:42.437040 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a53309f-edb3-4595-a564-b802339e15fa admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:16:42.447139 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a53309f-edb3-4595-a564-b802339e15fa admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:16:42.452472 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c30d5ad-cf63-4b33-b75f-542dcfddd06e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:16:42.453149 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c30d5ad-cf63-4b33-b75f-542dcfddd06e admin admin] REQUEST_METHOD: `PUT` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:42.453430 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c30d5ad-cf63-4b33-b75f-542dcfddd06e admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:42.453708 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c30d5ad-cf63-4b33-b75f-542dcfddd06e admin admin] PATH_INFO: `/v3/projects/e35a2e967e9b4ae1a342baac2bb50445/users/5aaaabf6958241bd9c06fcbaf3abaf2c/roles/9bd0470b544240209a420f640c79c06c` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:42.468187 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c30d5ad-cf63-4b33-b75f-542dcfddd06e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e35a2e967e9b4ae1a342baac2bb50445, user_id=5aaaabf6958241bd9c06fcbaf3abaf2c, role_id=9bd0470b544240209a420f640c79c06c)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:16:42.470876 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c30d5ad-cf63-4b33-b75f-542dcfddd06e admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:16:42.537010 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1268/2513] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:16:42 2022] PUT /identity/v3/projects/e35a2e967e9b4ae1a342baac2bb50445/users/5aaaabf6958241bd9c06fcbaf3abaf2c/roles/9bd0470b544240209a420f640c79c06c => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 03:16:42.547102 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3dd592-42a5-425e-a38f-f58c314f4e61 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:42.547387 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3dd592-42a5-425e-a38f-f58c314f4e61 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:42.547714 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3dd592-42a5-425e-a38f-f58c314f4e61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:42.576091 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-8c3dd592-42a5-425e-a38f-f58c314f4e61 None None] MFA Rules not processed for user `5aaaabf6958241bd9c06fcbaf3abaf2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:16:42.577295 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-8c3dd592-42a5-425e-a38f-f58c314f4e61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:16:42.643094 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1246/2514] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:16:42 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:16:42.653401 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c30d5ad-cf63-4b33-b75f-542dcfddd06e admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:16:42.655195 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-1c30d5ad-cf63-4b33-b75f-542dcfddd06e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:16:42.715388 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c30d5ad-cf63-4b33-b75f-542dcfddd06e admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:16:42.719145 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f212a14-63f3-432a-becd-2af3c7861ed2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a3cd1fbe6df48f8aa90d4ea69905e9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:16:42.719752 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f212a14-63f3-432a-becd-2af3c7861ed2 service glance] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:42.720032 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f212a14-63f3-432a-becd-2af3c7861ed2 service glance] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:42.720306 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f212a14-63f3-432a-becd-2af3c7861ed2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:42.729702 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f212a14-63f3-432a-becd-2af3c7861ed2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:16:42.732108 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f212a14-63f3-432a-becd-2af3c7861ed2 service glance] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:16:42.741437 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1269/2515] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:16:42 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:16:50.337316 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3dd592-42a5-425e-a38f-f58c314f4e61 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:16:50.339352 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-8c3dd592-42a5-425e-a38f-f58c314f4e61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:16:50.405173 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3dd592-42a5-425e-a38f-f58c314f4e61 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:16:50.409845 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb7e953-25c8-41c4-9455-c4c3a8b60c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:16:50.410479 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb7e953-25c8-41c4-9455-c4c3a8b60c19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:50.410753 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb7e953-25c8-41c4-9455-c4c3a8b60c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:50.411022 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb7e953-25c8-41c4-9455-c4c3a8b60c19 admin admin] PATH_INFO: `/v3/users/e7b3003e5d1f42b699e497ae54f934b5` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:50.412132 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb7e953-25c8-41c4-9455-c4c3a8b60c19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7b3003e5d1f42b699e497ae54f934b5)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:16:50.414167 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb7e953-25c8-41c4-9455-c4c3a8b60c19 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:16:50.622038 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-2eb7e953-25c8-41c4-9455-c4c3a8b60c19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7b3003e5d1f42b699e497ae54f934b5'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:50.629535 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-2eb7e953-25c8-41c4-9455-c4c3a8b60c19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7b3003e5d1f42b699e497ae54f934b5'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:50.636987 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:16:50.642421 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-2eb7e953-25c8-41c4-9455-c4c3a8b60c19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7b3003e5d1f42b699e497ae54f934b5'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:50.643150 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-2eb7e953-25c8-41c4-9455-c4c3a8b60c19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7b3003e5d1f42b699e497ae54f934b5'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:50.723677 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-2eb7e953-25c8-41c4-9455-c4c3a8b60c19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7b3003e5d1f42b699e497ae54f934b5'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:50.730075 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1247/2516] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:16:50 2022] DELETE /identity/v3/users/e7b3003e5d1f42b699e497ae54f934b5 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:16:50.735324 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724a0cb8-6dbd-49d0-b3f2-9852c01f4cad None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:50.736565 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724a0cb8-6dbd-49d0-b3f2-9852c01f4cad None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:50.736565 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724a0cb8-6dbd-49d0-b3f2-9852c01f4cad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:50.755999 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-724a0cb8-6dbd-49d0-b3f2-9852c01f4cad None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:16:50.759552 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-724a0cb8-6dbd-49d0-b3f2-9852c01f4cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:16:50.816861 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1270/2517] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:16:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:16:50.826663 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb7e953-25c8-41c4-9455-c4c3a8b60c19 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:16:50.828117 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-2eb7e953-25c8-41c4-9455-c4c3a8b60c19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:16:50.893978 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb7e953-25c8-41c4-9455-c4c3a8b60c19 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:16:50.896155 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b0603e-47de-4655-a040-7cbcceb0d13a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:16:50.896911 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b0603e-47de-4655-a040-7cbcceb0d13a service neutron] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:50.897197 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b0603e-47de-4655-a040-7cbcceb0d13a service neutron] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:50.897474 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b0603e-47de-4655-a040-7cbcceb0d13a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:50.907125 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b0603e-47de-4655-a040-7cbcceb0d13a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:16:50.908574 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b0603e-47de-4655-a040-7cbcceb0d13a service neutron] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:16:50.917848 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1248/2518] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:16:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:16:51.325334 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724a0cb8-6dbd-49d0-b3f2-9852c01f4cad None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:16:51.326915 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-724a0cb8-6dbd-49d0-b3f2-9852c01f4cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:16:51.346333 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724a0cb8-6dbd-49d0-b3f2-9852c01f4cad None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:16:51.351270 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d32275-de24-4aa3-bda9-d5bc00da9b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:16:51.351945 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d32275-de24-4aa3-bda9-d5bc00da9b97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:51.352262 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d32275-de24-4aa3-bda9-d5bc00da9b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:51.352584 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d32275-de24-4aa3-bda9-d5bc00da9b97 admin admin] PATH_INFO: `/v3/projects/2e2e309242af4ebf80efeacdc30d8152` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:51.353917 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d32275-de24-4aa3-bda9-d5bc00da9b97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e2e309242af4ebf80efeacdc30d8152)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:16:51.356480 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d32275-de24-4aa3-bda9-d5bc00da9b97 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:16:51.490991 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-a3d32275-de24-4aa3-bda9-d5bc00da9b97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e2e309242af4ebf80efeacdc30d8152', 'request_id': 'req-a3d32275-de24-4aa3-bda9-d5bc00da9b97'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:51.497014 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-a3d32275-de24-4aa3-bda9-d5bc00da9b97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e2e309242af4ebf80efeacdc30d8152', 'request_id': 'req-a3d32275-de24-4aa3-bda9-d5bc00da9b97'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:51.499664 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:INSERT=3,keystone:SELECT=93,keystone:DELETE=4 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:16:51.583267 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-a3d32275-de24-4aa3-bda9-d5bc00da9b97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e2e309242af4ebf80efeacdc30d8152', 'request_id': 'req-a3d32275-de24-4aa3-bda9-d5bc00da9b97'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:51.595417 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-a3d32275-de24-4aa3-bda9-d5bc00da9b97 admin admin] The token cache is being invalidate because project 2e2e309242af4ebf80efeacdc30d8152 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81517) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:16:51.595707 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-a3d32275-de24-4aa3-bda9-d5bc00da9b97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:51.596362 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-a3d32275-de24-4aa3-bda9-d5bc00da9b97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:16:51.597914 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1271/2519] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:16:51 2022] DELETE /identity/v3/projects/2e2e309242af4ebf80efeacdc30d8152 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:16:51.602889 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb60adaf-38b7-4b6b-be11-86d80c774f97 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:51.603372 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb60adaf-38b7-4b6b-be11-86d80c774f97 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:51.603505 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb60adaf-38b7-4b6b-be11-86d80c774f97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:51.623973 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-cb60adaf-38b7-4b6b-be11-86d80c774f97 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:16:51.625181 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-cb60adaf-38b7-4b6b-be11-86d80c774f97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:16:51.685167 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1249/2520] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:16:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:16:51.692935 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e1c335-6447-4233-9c08-ab38f02feb37 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:51.693269 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e1c335-6447-4233-9c08-ab38f02feb37 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:51.693576 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e1c335-6447-4233-9c08-ab38f02feb37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:51.714957 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-85e1c335-6447-4233-9c08-ab38f02feb37 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:16:51.716818 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-85e1c335-6447-4233-9c08-ab38f02feb37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:16:51.724792 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1272/2521] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:16:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:16:51.729841 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb60adaf-38b7-4b6b-be11-86d80c774f97 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:16:51.739938 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb60adaf-38b7-4b6b-be11-86d80c774f97 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:16:51.744885 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd6a7be-8414-4e2b-a4f6-eacb9ed398ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:16:51.745427 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd6a7be-8414-4e2b-a4f6-eacb9ed398ad admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:51.745710 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd6a7be-8414-4e2b-a4f6-eacb9ed398ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:51.746242 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd6a7be-8414-4e2b-a4f6-eacb9ed398ad admin admin] PATH_INFO: `/v3/domains` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:51.746817 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd6a7be-8414-4e2b-a4f6-eacb9ed398ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 03:16:51.747136 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd6a7be-8414-4e2b-a4f6-eacb9ed398ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:16:51.748742 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd6a7be-8414-4e2b-a4f6-eacb9ed398ad admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:16:51.760969 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1250/2522] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Oct 14 03:16:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 03:16:51.765801 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e1c335-6447-4233-9c08-ab38f02feb37 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:16:51.775123 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e1c335-6447-4233-9c08-ab38f02feb37 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:16:51.780379 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3248170-6fa4-4684-8bce-c6f5edbeb1aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:16:51.780970 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3248170-6fa4-4684-8bce-c6f5edbeb1aa admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:51.781255 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3248170-6fa4-4684-8bce-c6f5edbeb1aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:51.781527 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3248170-6fa4-4684-8bce-c6f5edbeb1aa admin admin] PATH_INFO: `/v3/projects` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:51.782195 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3248170-6fa4-4684-8bce-c6f5edbeb1aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:16:51.784093 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3248170-6fa4-4684-8bce-c6f5edbeb1aa admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:16:51.885465 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1273/2523] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:16:51 2022] POST /identity/v3/projects => generated 391 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:16:51.890206 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd6a7be-8414-4e2b-a4f6-eacb9ed398ad admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:16:51.900276 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd6a7be-8414-4e2b-a4f6-eacb9ed398ad admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:16:51.964321 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7a3438-b19d-4bc1-a2d1-a7f1a61fd2d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:16:51.964933 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7a3438-b19d-4bc1-a2d1-a7f1a61fd2d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:51.966148 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7a3438-b19d-4bc1-a2d1-a7f1a61fd2d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:51.966471 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7a3438-b19d-4bc1-a2d1-a7f1a61fd2d4 admin admin] PATH_INFO: `/v3/users` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:51.967123 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7a3438-b19d-4bc1-a2d1-a7f1a61fd2d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:16:51.969046 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7a3438-b19d-4bc1-a2d1-a7f1a61fd2d4 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:16:52.090132 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1251/2524] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:16:51 2022] POST /identity/v3/users => generated 296 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:16:52.094415 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3248170-6fa4-4684-8bce-c6f5edbeb1aa admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:16:52.104809 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3248170-6fa4-4684-8bce-c6f5edbeb1aa admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:16:52.109368 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd7b6a7-6cf9-4c68-914a-5c29b3119ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:16:52.110010 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd7b6a7-6cf9-4c68-914a-5c29b3119ec9 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:52.110295 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd7b6a7-6cf9-4c68-914a-5c29b3119ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:52.110602 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd7b6a7-6cf9-4c68-914a-5c29b3119ec9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:52.111393 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd7b6a7-6cf9-4c68-914a-5c29b3119ec9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:16:52.113377 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd7b6a7-6cf9-4c68-914a-5c29b3119ec9 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:16:52.124330 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1274/2525] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:16:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 03:16:52.128592 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7a3438-b19d-4bc1-a2d1-a7f1a61fd2d4 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:16:52.133881 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:16:52.140032 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7a3438-b19d-4bc1-a2d1-a7f1a61fd2d4 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:16:52.144710 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4c155d-8d38-406e-9d6e-c591a95ab6fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:16:52.145301 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4c155d-8d38-406e-9d6e-c591a95ab6fc admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:52.145579 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4c155d-8d38-406e-9d6e-c591a95ab6fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:52.145884 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4c155d-8d38-406e-9d6e-c591a95ab6fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:52.146538 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4c155d-8d38-406e-9d6e-c591a95ab6fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:16:52.148510 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4c155d-8d38-406e-9d6e-c591a95ab6fc admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:16:52.160304 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1252/2526] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:16:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:16:52.164053 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd7b6a7-6cf9-4c68-914a-5c29b3119ec9 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:16:52.174099 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd7b6a7-6cf9-4c68-914a-5c29b3119ec9 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:16:52.178653 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc27e867-510d-4cb0-a1be-11cfbb11ef20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:16:52.179297 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc27e867-510d-4cb0-a1be-11cfbb11ef20 admin admin] REQUEST_METHOD: `PUT` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:52.179577 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc27e867-510d-4cb0-a1be-11cfbb11ef20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:52.179855 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc27e867-510d-4cb0-a1be-11cfbb11ef20 admin admin] PATH_INFO: `/v3/projects/14b6f9e810df4b3c8b9dce38509e75e5/users/f6bf0e3b88864435bdf784f0b40a4ada/roles/9bd0470b544240209a420f640c79c06c` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:52.194921 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc27e867-510d-4cb0-a1be-11cfbb11ef20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14b6f9e810df4b3c8b9dce38509e75e5, user_id=f6bf0e3b88864435bdf784f0b40a4ada, role_id=9bd0470b544240209a420f640c79c06c)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:16:52.197598 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc27e867-510d-4cb0-a1be-11cfbb11ef20 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:16:52.258327 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1275/2527] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:16:52 2022] PUT /identity/v3/projects/14b6f9e810df4b3c8b9dce38509e75e5/users/f6bf0e3b88864435bdf784f0b40a4ada/roles/9bd0470b544240209a420f640c79c06c => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:16:52.270668 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ed480e-8e64-4a35-bc21-886cfd1305c9 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:52.270897 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ed480e-8e64-4a35-bc21-886cfd1305c9 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:52.271178 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ed480e-8e64-4a35-bc21-886cfd1305c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:52.298867 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-b6ed480e-8e64-4a35-bc21-886cfd1305c9 None None] MFA Rules not processed for user `f6bf0e3b88864435bdf784f0b40a4ada`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:16:52.300135 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-b6ed480e-8e64-4a35-bc21-886cfd1305c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:16:52.364290 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1253/2528] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:16:52 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 14 03:16:52.375481 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc27e867-510d-4cb0-a1be-11cfbb11ef20 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:16:52.377169 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-fc27e867-510d-4cb0-a1be-11cfbb11ef20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:16:52.436325 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc27e867-510d-4cb0-a1be-11cfbb11ef20 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:16:52.439966 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e3e7de-32d4-496b-b0cb-8223d6fe0d48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab599d98b0624a2e89090312bcab713f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:16:52.440566 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e3e7de-32d4-496b-b0cb-8223d6fe0d48 service cinder] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:16:52.440843 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e3e7de-32d4-496b-b0cb-8223d6fe0d48 service cinder] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:16:52.441114 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e3e7de-32d4-496b-b0cb-8223d6fe0d48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:16:52.449718 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e3e7de-32d4-496b-b0cb-8223d6fe0d48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:16:52.451213 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e3e7de-32d4-496b-b0cb-8223d6fe0d48 service cinder] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:16:52.459799 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1276/2529] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:16:52 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:17:02.361071 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=36 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:17:02.455865 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:DELETE=3,keystone:SELECT=57,keystone:INSERT=2 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:17:13.452711 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ed480e-8e64-4a35-bc21-886cfd1305c9 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:17:13.515385 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ed480e-8e64-4a35-bc21-886cfd1305c9 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:17:13.519813 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c6dd10-49b7-4d0b-9db7-5c2ef8d32e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:17:13.520466 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c6dd10-49b7-4d0b-9db7-5c2ef8d32e37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:17:13.522262 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c6dd10-49b7-4d0b-9db7-5c2ef8d32e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:17:13.522262 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c6dd10-49b7-4d0b-9db7-5c2ef8d32e37 admin admin] PATH_INFO: `/v3/users/f6bf0e3b88864435bdf784f0b40a4ada` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:17:13.522262 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c6dd10-49b7-4d0b-9db7-5c2ef8d32e37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6bf0e3b88864435bdf784f0b40a4ada)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:17:13.524038 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c6dd10-49b7-4d0b-9db7-5c2ef8d32e37 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:17:13.656013 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-43c6dd10-49b7-4d0b-9db7-5c2ef8d32e37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6bf0e3b88864435bdf784f0b40a4ada'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:17:13.662824 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-43c6dd10-49b7-4d0b-9db7-5c2ef8d32e37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6bf0e3b88864435bdf784f0b40a4ada'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:17:13.675336 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-43c6dd10-49b7-4d0b-9db7-5c2ef8d32e37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6bf0e3b88864435bdf784f0b40a4ada'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:17:13.676007 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-43c6dd10-49b7-4d0b-9db7-5c2ef8d32e37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6bf0e3b88864435bdf784f0b40a4ada'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:17:13.757731 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-43c6dd10-49b7-4d0b-9db7-5c2ef8d32e37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6bf0e3b88864435bdf784f0b40a4ada'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:17:13.764779 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1254/2530] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:17:13 2022] DELETE /identity/v3/users/f6bf0e3b88864435bdf784f0b40a4ada => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:17:13.769278 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8619dee3-7cc5-49f5-a282-bfb5bf6cd20b None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:17:13.769780 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8619dee3-7cc5-49f5-a282-bfb5bf6cd20b None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:17:13.769886 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8619dee3-7cc5-49f5-a282-bfb5bf6cd20b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:17:13.789076 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-8619dee3-7cc5-49f5-a282-bfb5bf6cd20b None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:17:13.790331 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-8619dee3-7cc5-49f5-a282-bfb5bf6cd20b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:17:13.848215 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1277/2531] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:17:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:17:13.857822 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c6dd10-49b7-4d0b-9db7-5c2ef8d32e37 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:17:13.859791 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-43c6dd10-49b7-4d0b-9db7-5c2ef8d32e37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:17:13.915996 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c6dd10-49b7-4d0b-9db7-5c2ef8d32e37 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:17:13.919738 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cea5498-8587-440e-b54d-def05406053b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:17:13.920368 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cea5498-8587-440e-b54d-def05406053b service neutron] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:17:13.921257 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cea5498-8587-440e-b54d-def05406053b service neutron] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:17:13.921558 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cea5498-8587-440e-b54d-def05406053b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:17:13.930459 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cea5498-8587-440e-b54d-def05406053b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:17:13.931873 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cea5498-8587-440e-b54d-def05406053b service neutron] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:17:13.941162 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1255/2532] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:17:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 03:17:14.331225 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8619dee3-7cc5-49f5-a282-bfb5bf6cd20b None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:17:14.332678 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-8619dee3-7cc5-49f5-a282-bfb5bf6cd20b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:17:14.345427 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8619dee3-7cc5-49f5-a282-bfb5bf6cd20b None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:17:14.350105 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b468d08e-6272-490c-a521-e77d08c22499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:17:14.350733 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b468d08e-6272-490c-a521-e77d08c22499 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:17:14.351012 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b468d08e-6272-490c-a521-e77d08c22499 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:17:14.351285 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b468d08e-6272-490c-a521-e77d08c22499 admin admin] PATH_INFO: `/v3/projects/14b6f9e810df4b3c8b9dce38509e75e5` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:17:14.352411 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b468d08e-6272-490c-a521-e77d08c22499 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14b6f9e810df4b3c8b9dce38509e75e5)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:17:14.354338 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b468d08e-6272-490c-a521-e77d08c22499 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:17:14.462981 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-b468d08e-6272-490c-a521-e77d08c22499 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14b6f9e810df4b3c8b9dce38509e75e5', 'request_id': 'req-b468d08e-6272-490c-a521-e77d08c22499'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:17:14.469941 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-b468d08e-6272-490c-a521-e77d08c22499 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14b6f9e810df4b3c8b9dce38509e75e5', 'request_id': 'req-b468d08e-6272-490c-a521-e77d08c22499'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:17:14.592017 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-b468d08e-6272-490c-a521-e77d08c22499 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14b6f9e810df4b3c8b9dce38509e75e5', 'request_id': 'req-b468d08e-6272-490c-a521-e77d08c22499'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:17:14.602954 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-b468d08e-6272-490c-a521-e77d08c22499 admin admin] The token cache is being invalidate because project 14b6f9e810df4b3c8b9dce38509e75e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81517) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:17:14.603242 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-b468d08e-6272-490c-a521-e77d08c22499 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:17:14.603859 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-b468d08e-6272-490c-a521-e77d08c22499 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:17:14.605377 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1278/2533] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:17:14 2022] DELETE /identity/v3/projects/14b6f9e810df4b3c8b9dce38509e75e5 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:17:14.610615 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c797b15a-8d60-4a68-a0ff-9113365e98a2 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:17:14.611091 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c797b15a-8d60-4a68-a0ff-9113365e98a2 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:17:14.611616 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c797b15a-8d60-4a68-a0ff-9113365e98a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:17:14.631109 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-c797b15a-8d60-4a68-a0ff-9113365e98a2 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:17:14.632304 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-c797b15a-8d60-4a68-a0ff-9113365e98a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:17:14.649353 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:17:14.690891 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1256/2534] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:17:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:17:14.698640 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c95ec-db4e-4d73-bf28-2eed30a6aaf0 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:17:14.698927 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c95ec-db4e-4d73-bf28-2eed30a6aaf0 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:17:14.699206 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c95ec-db4e-4d73-bf28-2eed30a6aaf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:17:14.719194 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-088c95ec-db4e-4d73-bf28-2eed30a6aaf0 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:17:14.720391 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-088c95ec-db4e-4d73-bf28-2eed30a6aaf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:17:14.728168 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1279/2535] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:17:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:17:14.732339 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c797b15a-8d60-4a68-a0ff-9113365e98a2 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:17:14.742473 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c797b15a-8d60-4a68-a0ff-9113365e98a2 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:17:14.747262 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365ce3af-d0d6-469e-b8e3-326fa73555e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:17:14.747897 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365ce3af-d0d6-469e-b8e3-326fa73555e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:17:14.748308 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365ce3af-d0d6-469e-b8e3-326fa73555e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:17:14.749078 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365ce3af-d0d6-469e-b8e3-326fa73555e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:17:14.749732 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365ce3af-d0d6-469e-b8e3-326fa73555e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81518) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 03:17:14.750266 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365ce3af-d0d6-469e-b8e3-326fa73555e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:17:14.752009 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365ce3af-d0d6-469e-b8e3-326fa73555e8 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:17:14.764323 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1257/2536] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Oct 14 03:17:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 03:17:14.769444 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088c95ec-db4e-4d73-bf28-2eed30a6aaf0 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:17:14.778606 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088c95ec-db4e-4d73-bf28-2eed30a6aaf0 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:17:14.783328 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aea76f0-07d1-4999-913e-362042a4627e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:17:14.786383 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aea76f0-07d1-4999-913e-362042a4627e admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:17:14.786383 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aea76f0-07d1-4999-913e-362042a4627e admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:17:14.786383 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aea76f0-07d1-4999-913e-362042a4627e admin admin] PATH_INFO: `/v3/projects` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:17:14.786383 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aea76f0-07d1-4999-913e-362042a4627e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:17:14.787054 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aea76f0-07d1-4999-913e-362042a4627e admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:17:14.896099 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1280/2537] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:17:14 2022] POST /identity/v3/projects => generated 383 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:17:14.900332 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365ce3af-d0d6-469e-b8e3-326fa73555e8 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:17:14.910320 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365ce3af-d0d6-469e-b8e3-326fa73555e8 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:17:14.964902 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de2a009-0d1d-4c81-950b-7c8b560356dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:17:14.965489 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de2a009-0d1d-4c81-950b-7c8b560356dd admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:17:14.965785 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de2a009-0d1d-4c81-950b-7c8b560356dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:17:14.966061 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de2a009-0d1d-4c81-950b-7c8b560356dd admin admin] PATH_INFO: `/v3/users` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:17:14.966692 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de2a009-0d1d-4c81-950b-7c8b560356dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:17:14.968775 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de2a009-0d1d-4c81-950b-7c8b560356dd admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:17:15.046244 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1258/2538] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:17:14 2022] POST /identity/v3/users => generated 292 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:17:15.050320 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aea76f0-07d1-4999-913e-362042a4627e admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:17:15.060483 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aea76f0-07d1-4999-913e-362042a4627e admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:17:15.065596 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332934a3-44e6-42d3-8d01-5be5b58de0e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:17:15.066223 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332934a3-44e6-42d3-8d01-5be5b58de0e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:17:15.066502 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332934a3-44e6-42d3-8d01-5be5b58de0e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:17:15.066770 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332934a3-44e6-42d3-8d01-5be5b58de0e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:17:15.067409 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332934a3-44e6-42d3-8d01-5be5b58de0e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:17:15.068998 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332934a3-44e6-42d3-8d01-5be5b58de0e6 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:17:15.078679 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1281/2539] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:17:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:17:15.082573 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de2a009-0d1d-4c81-950b-7c8b560356dd admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:17:15.092453 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de2a009-0d1d-4c81-950b-7c8b560356dd admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:17:15.096979 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efaa954-a79e-4778-8343-9f5b924b6a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:17:15.097561 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efaa954-a79e-4778-8343-9f5b924b6a5e admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:17:15.097873 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efaa954-a79e-4778-8343-9f5b924b6a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:17:15.098148 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efaa954-a79e-4778-8343-9f5b924b6a5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:17:15.098777 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efaa954-a79e-4778-8343-9f5b924b6a5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:17:15.100356 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efaa954-a79e-4778-8343-9f5b924b6a5e admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:17:15.111105 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1259/2540] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:17:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:17:15.115165 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332934a3-44e6-42d3-8d01-5be5b58de0e6 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:17:15.125547 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332934a3-44e6-42d3-8d01-5be5b58de0e6 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:17:15.130130 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ec33ba-c418-40ef-957b-4049727164a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:17:15.130764 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ec33ba-c418-40ef-957b-4049727164a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:17:15.131040 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ec33ba-c418-40ef-957b-4049727164a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:17:15.131313 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ec33ba-c418-40ef-957b-4049727164a6 admin admin] PATH_INFO: `/v3/projects/f2c0270c3f82423ea62db21e22ac86bd/users/ca9545f301e24e579903df7f87337f8e/roles/9bd0470b544240209a420f640c79c06c` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:17:15.145714 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ec33ba-c418-40ef-957b-4049727164a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2c0270c3f82423ea62db21e22ac86bd, user_id=ca9545f301e24e579903df7f87337f8e, role_id=9bd0470b544240209a420f640c79c06c)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:17:15.148436 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ec33ba-c418-40ef-957b-4049727164a6 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:17:15.243508 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1282/2541] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:17:15 2022] PUT /identity/v3/projects/f2c0270c3f82423ea62db21e22ac86bd/users/ca9545f301e24e579903df7f87337f8e/roles/9bd0470b544240209a420f640c79c06c => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 03:17:15.248436 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49210571-898c-471e-9cf6-418bf978e30a None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:17:15.248723 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49210571-898c-471e-9cf6-418bf978e30a None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:17:15.248998 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49210571-898c-471e-9cf6-418bf978e30a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:17:15.267737 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-49210571-898c-471e-9cf6-418bf978e30a None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:17:15.268959 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-49210571-898c-471e-9cf6-418bf978e30a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:17:15.326542 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1260/2542] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:17:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:17:15.336291 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ec33ba-c418-40ef-957b-4049727164a6 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:17:15.338033 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-70ec33ba-c418-40ef-957b-4049727164a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:17:15.361227 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:17:15.396939 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ec33ba-c418-40ef-957b-4049727164a6 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:17:15.400858 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3ad606-96c7-43b1-9f8b-a8cbe31ea18c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:17:15.401384 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3ad606-96c7-43b1-9f8b-a8cbe31ea18c service neutron] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:17:15.401669 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3ad606-96c7-43b1-9f8b-a8cbe31ea18c service neutron] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:17:15.401978 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3ad606-96c7-43b1-9f8b-a8cbe31ea18c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:17:15.411380 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3ad606-96c7-43b1-9f8b-a8cbe31ea18c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:17:15.412815 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3ad606-96c7-43b1-9f8b-a8cbe31ea18c service neutron] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:17:15.422330 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1283/2543] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:17:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:17:20.743678 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23d30df-ee31-44e3-9474-3aebbae98730 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:17:20.744210 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23d30df-ee31-44e3-9474-3aebbae98730 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:17:20.744210 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23d30df-ee31-44e3-9474-3aebbae98730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:17:20.773584 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-a23d30df-ee31-44e3-9474-3aebbae98730 None None] MFA Rules not processed for user `ca9545f301e24e579903df7f87337f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:17:20.774981 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-a23d30df-ee31-44e3-9474-3aebbae98730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:17:20.796400 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:17:20.866021 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1261/2544] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:17:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:17:20.866145 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3ad606-96c7-43b1-9f8b-a8cbe31ea18c service neutron] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:17:20.875554 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3ad606-96c7-43b1-9f8b-a8cbe31ea18c service neutron] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:17:20.880230 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a046aa-b728-4b38-9595-ddf0e2a23ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:17:20.880841 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a046aa-b728-4b38-9595-ddf0e2a23ee3 admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:17:20.881115 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a046aa-b728-4b38-9595-ddf0e2a23ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:17:20.881435 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a046aa-b728-4b38-9595-ddf0e2a23ee3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:17:20.882167 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a046aa-b728-4b38-9595-ddf0e2a23ee3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:17:20.884065 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a046aa-b728-4b38-9595-ddf0e2a23ee3 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:17:20.968480 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1284/2545] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:17:20 2022] POST /identity/v3/projects => generated 385 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:17:20.973404 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23d30df-ee31-44e3-9474-3aebbae98730 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:17:20.987178 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23d30df-ee31-44e3-9474-3aebbae98730 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:17:21.048420 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372f3d29-02bb-4e17-aab3-c8fe7335e207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:17:21.049026 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372f3d29-02bb-4e17-aab3-c8fe7335e207 admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:17:21.049307 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372f3d29-02bb-4e17-aab3-c8fe7335e207 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:17:21.049579 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372f3d29-02bb-4e17-aab3-c8fe7335e207 admin admin] PATH_INFO: `/v3/users` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:17:21.050252 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372f3d29-02bb-4e17-aab3-c8fe7335e207 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:17:21.052194 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372f3d29-02bb-4e17-aab3-c8fe7335e207 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:17:21.148566 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1262/2546] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:17:20 2022] POST /identity/v3/users => generated 299 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:17:21.152609 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a046aa-b728-4b38-9595-ddf0e2a23ee3 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:17:21.163592 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a046aa-b728-4b38-9595-ddf0e2a23ee3 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:17:21.168221 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d330bd-681a-45d3-92e7-8d6412db7476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:17:21.168826 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d330bd-681a-45d3-92e7-8d6412db7476 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:17:21.169115 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d330bd-681a-45d3-92e7-8d6412db7476 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:17:21.169391 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d330bd-681a-45d3-92e7-8d6412db7476 admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:17:21.170070 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d330bd-681a-45d3-92e7-8d6412db7476 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:17:21.171680 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d330bd-681a-45d3-92e7-8d6412db7476 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:17:21.182193 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1285/2547] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:17:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:17:21.186377 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372f3d29-02bb-4e17-aab3-c8fe7335e207 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:17:21.197330 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372f3d29-02bb-4e17-aab3-c8fe7335e207 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:17:21.202895 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26033ce-f260-4d3d-9023-62b1e13ba343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:17:21.203564 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26033ce-f260-4d3d-9023-62b1e13ba343 admin admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:17:21.203844 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26033ce-f260-4d3d-9023-62b1e13ba343 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:17:21.204125 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26033ce-f260-4d3d-9023-62b1e13ba343 admin admin] PATH_INFO: `/v3/projects/0155abbda1374196b19097d189a0a1f2/users/42d9de861bef4f6caaafd28b9333cd14/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:17:21.221472 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26033ce-f260-4d3d-9023-62b1e13ba343 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0155abbda1374196b19097d189a0a1f2, user_id=42d9de861bef4f6caaafd28b9333cd14, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:17:21.224086 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26033ce-f260-4d3d-9023-62b1e13ba343 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:17:21.314577 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1263/2548] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:17:21 2022] PUT /identity/v3/projects/0155abbda1374196b19097d189a0a1f2/users/42d9de861bef4f6caaafd28b9333cd14/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 03:17:21.318782 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d330bd-681a-45d3-92e7-8d6412db7476 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:17:21.329657 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d330bd-681a-45d3-92e7-8d6412db7476 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:17:21.386342 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c000c512-8ba8-4841-af26-54f4b00acabd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:17:21.387145 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c000c512-8ba8-4841-af26-54f4b00acabd admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:17:21.389685 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c000c512-8ba8-4841-af26-54f4b00acabd admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:17:21.389685 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c000c512-8ba8-4841-af26-54f4b00acabd admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:17:21.389685 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c000c512-8ba8-4841-af26-54f4b00acabd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:17:21.390741 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c000c512-8ba8-4841-af26-54f4b00acabd admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:17:21.401825 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1286/2549] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:17:21 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:17:21.407382 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26033ce-f260-4d3d-9023-62b1e13ba343 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:17:21.418533 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26033ce-f260-4d3d-9023-62b1e13ba343 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:17:21.424168 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d7eaa3-7f21-487d-8319-8c9d9f05b886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:17:21.424840 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d7eaa3-7f21-487d-8319-8c9d9f05b886 admin admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:17:21.425183 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d7eaa3-7f21-487d-8319-8c9d9f05b886 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:17:21.425450 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d7eaa3-7f21-487d-8319-8c9d9f05b886 admin admin] PATH_INFO: `/v3/projects/0155abbda1374196b19097d189a0a1f2/users/42d9de861bef4f6caaafd28b9333cd14/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:17:21.427482 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d7eaa3-7f21-487d-8319-8c9d9f05b886 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0155abbda1374196b19097d189a0a1f2, user_id=42d9de861bef4f6caaafd28b9333cd14, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:17:21.430197 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d7eaa3-7f21-487d-8319-8c9d9f05b886 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:17:21.442011 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1264/2550] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:17:21 2022] PUT /identity/v3/projects/0155abbda1374196b19097d189a0a1f2/users/42d9de861bef4f6caaafd28b9333cd14/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:17:21.896016 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c000c512-8ba8-4841-af26-54f4b00acabd admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:17:21.906263 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-c000c512-8ba8-4841-af26-54f4b00acabd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:17:21.916286 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c000c512-8ba8-4841-af26-54f4b00acabd admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:17:21.991967 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30268195-dc27-468b-ad99-18219d77ce58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:17:21.992696 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30268195-dc27-468b-ad99-18219d77ce58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:17:21.993051 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30268195-dc27-468b-ad99-18219d77ce58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:17:21.993410 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30268195-dc27-468b-ad99-18219d77ce58 admin admin] PATH_INFO: `/v3/users/5aaaabf6958241bd9c06fcbaf3abaf2c` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:17:21.994654 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30268195-dc27-468b-ad99-18219d77ce58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5aaaabf6958241bd9c06fcbaf3abaf2c)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:17:21.996775 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30268195-dc27-468b-ad99-18219d77ce58 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:17:22.138695 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:17:22.253426 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-30268195-dc27-468b-ad99-18219d77ce58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5aaaabf6958241bd9c06fcbaf3abaf2c'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:17:22.254360 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-30268195-dc27-468b-ad99-18219d77ce58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5aaaabf6958241bd9c06fcbaf3abaf2c'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:17:22.337228 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-30268195-dc27-468b-ad99-18219d77ce58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5aaaabf6958241bd9c06fcbaf3abaf2c'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:17:22.337909 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-30268195-dc27-468b-ad99-18219d77ce58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5aaaabf6958241bd9c06fcbaf3abaf2c'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:17:22.345792 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-30268195-dc27-468b-ad99-18219d77ce58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5aaaabf6958241bd9c06fcbaf3abaf2c'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:17:22.365231 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1287/2551] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:17:21 2022] DELETE /identity/v3/users/5aaaabf6958241bd9c06fcbaf3abaf2c => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:17:22.369569 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704b0f37-6dba-4b28-920b-61c0b97ee3ac None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:17:22.370704 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704b0f37-6dba-4b28-920b-61c0b97ee3ac None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:17:22.370987 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704b0f37-6dba-4b28-920b-61c0b97ee3ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:17:22.390268 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-704b0f37-6dba-4b28-920b-61c0b97ee3ac None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:17:22.391573 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-704b0f37-6dba-4b28-920b-61c0b97ee3ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:17:22.457280 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1265/2552] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:17:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:17:22.469532 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30268195-dc27-468b-ad99-18219d77ce58 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:17:22.471910 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-30268195-dc27-468b-ad99-18219d77ce58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:17:22.542909 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30268195-dc27-468b-ad99-18219d77ce58 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:17:22.546505 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e97b11c-d95a-4134-9ab4-c9954207239f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:17:22.547202 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e97b11c-d95a-4134-9ab4-c9954207239f service neutron] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:17:22.547459 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e97b11c-d95a-4134-9ab4-c9954207239f service neutron] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:17:22.547737 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e97b11c-d95a-4134-9ab4-c9954207239f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:17:22.557721 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e97b11c-d95a-4134-9ab4-c9954207239f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:17:22.559523 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e97b11c-d95a-4134-9ab4-c9954207239f service neutron] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:17:22.570840 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1288/2553] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:17:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:17:23.065673 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704b0f37-6dba-4b28-920b-61c0b97ee3ac None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:17:23.067275 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-704b0f37-6dba-4b28-920b-61c0b97ee3ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:17:23.081153 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704b0f37-6dba-4b28-920b-61c0b97ee3ac None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:17:23.089286 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dbd56a-8755-425e-9c14-7e3f3fa5d6c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:17:23.090242 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dbd56a-8755-425e-9c14-7e3f3fa5d6c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:17:23.091181 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dbd56a-8755-425e-9c14-7e3f3fa5d6c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:17:23.091650 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dbd56a-8755-425e-9c14-7e3f3fa5d6c3 admin admin] PATH_INFO: `/v3/projects/e35a2e967e9b4ae1a342baac2bb50445` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:17:23.093078 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dbd56a-8755-425e-9c14-7e3f3fa5d6c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e35a2e967e9b4ae1a342baac2bb50445)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:17:23.095314 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dbd56a-8755-425e-9c14-7e3f3fa5d6c3 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:17:23.103087 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:17:23.221683 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-06dbd56a-8755-425e-9c14-7e3f3fa5d6c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e35a2e967e9b4ae1a342baac2bb50445', 'request_id': 'req-06dbd56a-8755-425e-9c14-7e3f3fa5d6c3'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:17:23.228508 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-06dbd56a-8755-425e-9c14-7e3f3fa5d6c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e35a2e967e9b4ae1a342baac2bb50445', 'request_id': 'req-06dbd56a-8755-425e-9c14-7e3f3fa5d6c3'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:17:23.300595 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-06dbd56a-8755-425e-9c14-7e3f3fa5d6c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e35a2e967e9b4ae1a342baac2bb50445', 'request_id': 'req-06dbd56a-8755-425e-9c14-7e3f3fa5d6c3'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:17:23.313994 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-06dbd56a-8755-425e-9c14-7e3f3fa5d6c3 admin admin] The token cache is being invalidate because project e35a2e967e9b4ae1a342baac2bb50445 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:17:23.314389 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-06dbd56a-8755-425e-9c14-7e3f3fa5d6c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:17:23.315120 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-06dbd56a-8755-425e-9c14-7e3f3fa5d6c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:17:23.317296 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1266/2554] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:17:23 2022] DELETE /identity/v3/projects/e35a2e967e9b4ae1a342baac2bb50445 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:17:27.043109 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa7b03c-5b5b-4a99-804d-4b54802e2d1d None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:17:27.043641 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa7b03c-5b5b-4a99-804d-4b54802e2d1d None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:17:27.043641 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa7b03c-5b5b-4a99-804d-4b54802e2d1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:17:27.073183 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-eaa7b03c-5b5b-4a99-804d-4b54802e2d1d None None] MFA Rules not processed for user `42d9de861bef4f6caaafd28b9333cd14`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:17:27.074529 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-eaa7b03c-5b5b-4a99-804d-4b54802e2d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:17:27.154696 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1289/2555] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:17:27 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:17:27.163230 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dbd56a-8755-425e-9c14-7e3f3fa5d6c3 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:17:27.164728 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-06dbd56a-8755-425e-9c14-7e3f3fa5d6c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:17:27.226257 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dbd56a-8755-425e-9c14-7e3f3fa5d6c3 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:17:27.229881 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6014b33-c991-488d-8678-37b1ecefa501 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a3cd1fbe6df48f8aa90d4ea69905e9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:17:27.230497 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6014b33-c991-488d-8678-37b1ecefa501 service glance] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:17:27.230781 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6014b33-c991-488d-8678-37b1ecefa501 service glance] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:17:27.231059 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6014b33-c991-488d-8678-37b1ecefa501 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:17:27.232962 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-e6014b33-c991-488d-8678-37b1ecefa501 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:17:27.293603 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6014b33-c991-488d-8678-37b1ecefa501 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:17:27.295350 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6014b33-c991-488d-8678-37b1ecefa501 service glance] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:17:27.306253 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1267/2556] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:17:27 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:17:27.594695 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c1e6f9-09bf-4108-bbc5-aa1416413b03 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:17:27.594987 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c1e6f9-09bf-4108-bbc5-aa1416413b03 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:17:27.595269 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c1e6f9-09bf-4108-bbc5-aa1416413b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:17:27.615626 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-93c1e6f9-09bf-4108-bbc5-aa1416413b03 None None] MFA Rules not processed for user `44b37050076b42e6af1a67bd7d6d06e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:17:27.616853 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-93c1e6f9-09bf-4108-bbc5-aa1416413b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:17:27.646755 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:17:27.672417 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1290/2557] 10.222.0.39 () {58 vars in 1009 bytes} [Fri Oct 14 03:17:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:17:27.801919 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6014b33-c991-488d-8678-37b1ecefa501 service glance] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:17:27.803328 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-e6014b33-c991-488d-8678-37b1ecefa501 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:17:27.854655 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:17:27.880453 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6014b33-c991-488d-8678-37b1ecefa501 service glance] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:17:27.886654 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073bd452-7228-46d6-a895-9d871ee363ef service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5e07da64804243bc18862072b027a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:17:27.887362 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073bd452-7228-46d6-a895-9d871ee363ef service swift] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:17:27.888262 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073bd452-7228-46d6-a895-9d871ee363ef service swift] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:17:27.889986 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073bd452-7228-46d6-a895-9d871ee363ef service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:17:27.903728 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073bd452-7228-46d6-a895-9d871ee363ef service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:17:27.903728 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073bd452-7228-46d6-a895-9d871ee363ef service swift] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:17:27.910406 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1268/2558] 10.222.0.39 () {60 vars in 1384 bytes} [Fri Oct 14 03:17:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 14 03:17:37.735136 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=10 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:17:37.959498 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=10 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:17:38.013346 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383753f5-8c2d-4be9-a070-5dcc8e18d588 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:17:38.013346 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383753f5-8c2d-4be9-a070-5dcc8e18d588 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:17:38.013750 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383753f5-8c2d-4be9-a070-5dcc8e18d588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:17:38.033088 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-383753f5-8c2d-4be9-a070-5dcc8e18d588 None None] MFA Rules not processed for user `ca9545f301e24e579903df7f87337f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:17:38.034305 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-383753f5-8c2d-4be9-a070-5dcc8e18d588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:17:38.041072 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1291/2559] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:17:38 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:17:38.079616 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073bd452-7228-46d6-a895-9d871ee363ef service swift] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:17:38.089083 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073bd452-7228-46d6-a895-9d871ee363ef service swift] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:17:38.092698 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654803b7-69c5-4eb9-b274-0b0f805da004 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a3cd1fbe6df48f8aa90d4ea69905e9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:17:38.093331 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654803b7-69c5-4eb9-b274-0b0f805da004 service glance] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:17:38.093614 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654803b7-69c5-4eb9-b274-0b0f805da004 service glance] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:17:38.093933 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654803b7-69c5-4eb9-b274-0b0f805da004 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:17:38.095689 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-654803b7-69c5-4eb9-b274-0b0f805da004 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:17:38.165887 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654803b7-69c5-4eb9-b274-0b0f805da004 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:17:38.167320 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654803b7-69c5-4eb9-b274-0b0f805da004 service glance] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:17:38.178647 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1269/2560] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:17:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:17:38.779849 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383753f5-8c2d-4be9-a070-5dcc8e18d588 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:17:38.783095 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-383753f5-8c2d-4be9-a070-5dcc8e18d588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:17:38.840595 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383753f5-8c2d-4be9-a070-5dcc8e18d588 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:17:38.844887 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843a83c7-4f65-4ead-96cb-a8acee4b69ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:17:38.845506 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a83c7-4f65-4ead-96cb-a8acee4b69ad service neutron] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:17:38.845808 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a83c7-4f65-4ead-96cb-a8acee4b69ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:17:38.846088 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a83c7-4f65-4ead-96cb-a8acee4b69ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:17:38.848891 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-843a83c7-4f65-4ead-96cb-a8acee4b69ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:17:38.863576 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843a83c7-4f65-4ead-96cb-a8acee4b69ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:17:38.865173 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843a83c7-4f65-4ead-96cb-a8acee4b69ad service neutron] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:17:38.876653 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1292/2561] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:17:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:17:41.943707 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46424a9e-f62c-42f6-b65a-6090390b9808 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:17:41.944220 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46424a9e-f62c-42f6-b65a-6090390b9808 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:17:41.944322 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46424a9e-f62c-42f6-b65a-6090390b9808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:17:41.963767 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-46424a9e-f62c-42f6-b65a-6090390b9808 None None] MFA Rules not processed for user `44b37050076b42e6af1a67bd7d6d06e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:17:41.964972 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-46424a9e-f62c-42f6-b65a-6090390b9808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:17:41.974848 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1270/2562] 10.222.0.39 () {58 vars in 1009 bytes} [Fri Oct 14 03:17:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:17:41.986904 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843a83c7-4f65-4ead-96cb-a8acee4b69ad service neutron] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:17:41.999040 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843a83c7-4f65-4ead-96cb-a8acee4b69ad service neutron] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:17:42.004513 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7ed101-974f-4cfc-96da-9b2ea726bd90 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5e07da64804243bc18862072b027a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:17:42.005268 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7ed101-974f-4cfc-96da-9b2ea726bd90 service swift] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:17:42.005651 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7ed101-974f-4cfc-96da-9b2ea726bd90 service swift] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:17:42.006056 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7ed101-974f-4cfc-96da-9b2ea726bd90 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:17:42.015668 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7ed101-974f-4cfc-96da-9b2ea726bd90 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:17:42.017559 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7ed101-974f-4cfc-96da-9b2ea726bd90 service swift] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:17:42.026954 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1293/2563] 10.222.0.39 () {60 vars in 1384 bytes} [Fri Oct 14 03:17:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 14 03:17:51.958169 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=37 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:17:52.023459 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=35 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:18:09.784819 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46424a9e-f62c-42f6-b65a-6090390b9808 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:18:09.786320 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-46424a9e-f62c-42f6-b65a-6090390b9808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:18:09.797552 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46424a9e-f62c-42f6-b65a-6090390b9808 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:18:09.801607 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238bd89d-dabf-4a9c-af88-6e93da012e0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:18:09.802236 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238bd89d-dabf-4a9c-af88-6e93da012e0c service neutron] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:18:09.802518 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238bd89d-dabf-4a9c-af88-6e93da012e0c service neutron] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:18:09.802791 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238bd89d-dabf-4a9c-af88-6e93da012e0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:18:09.811590 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238bd89d-dabf-4a9c-af88-6e93da012e0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:18:09.812999 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238bd89d-dabf-4a9c-af88-6e93da012e0c service neutron] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:18:09.822929 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1271/2564] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:18:09 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:18:11.654918 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7ed101-974f-4cfc-96da-9b2ea726bd90 service swift] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:18:11.657270 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-4f7ed101-974f-4cfc-96da-9b2ea726bd90 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:18:11.676778 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7ed101-974f-4cfc-96da-9b2ea726bd90 service swift] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:18:11.682502 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69beaa75-4daa-4200-82ab-da16be1c7f82 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b07bf0faca47b28b50990d1720853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:18:11.683194 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69beaa75-4daa-4200-82ab-da16be1c7f82 service nova] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:18:11.683539 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69beaa75-4daa-4200-82ab-da16be1c7f82 service nova] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:18:11.683822 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69beaa75-4daa-4200-82ab-da16be1c7f82 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:18:11.685583 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-69beaa75-4daa-4200-82ab-da16be1c7f82 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:18:11.698684 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69beaa75-4daa-4200-82ab-da16be1c7f82 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:18:11.700271 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69beaa75-4daa-4200-82ab-da16be1c7f82 service nova] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:18:11.711650 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1294/2565] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:18:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:18:11.816641 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238bd89d-dabf-4a9c-af88-6e93da012e0c service neutron] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:18:11.826970 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238bd89d-dabf-4a9c-af88-6e93da012e0c service neutron] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:18:11.831843 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362a4bcd-8304-4919-b85f-87d5ca535d2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:18:11.832536 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362a4bcd-8304-4919-b85f-87d5ca535d2f service neutron] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:18:11.832943 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362a4bcd-8304-4919-b85f-87d5ca535d2f service neutron] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:18:11.833262 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362a4bcd-8304-4919-b85f-87d5ca535d2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:18:11.835048 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-362a4bcd-8304-4919-b85f-87d5ca535d2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:18:11.845217 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362a4bcd-8304-4919-b85f-87d5ca535d2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:18:11.846670 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362a4bcd-8304-4919-b85f-87d5ca535d2f service neutron] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:18:11.855497 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1272/2566] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:18:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:18:14.593072 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69beaa75-4daa-4200-82ab-da16be1c7f82 service nova] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:18:14.605342 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69beaa75-4daa-4200-82ab-da16be1c7f82 service nova] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:18:14.612965 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ef3784-5cd6-442e-95d7-94e9744ceb30 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b07bf0faca47b28b50990d1720853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:18:14.615391 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ef3784-5cd6-442e-95d7-94e9744ceb30 service nova] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:18:14.615391 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ef3784-5cd6-442e-95d7-94e9744ceb30 service nova] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:18:14.615391 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ef3784-5cd6-442e-95d7-94e9744ceb30 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:18:14.617792 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-63ef3784-5cd6-442e-95d7-94e9744ceb30 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:18:14.634936 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ef3784-5cd6-442e-95d7-94e9744ceb30 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:18:14.636588 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ef3784-5cd6-442e-95d7-94e9744ceb30 service nova] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:18:14.649521 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1295/2567] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:18:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:18:21.851599 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=12 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:18:24.643362 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=12 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:19:14.259003 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362a4bcd-8304-4919-b85f-87d5ca535d2f service neutron] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:19:14.272138 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362a4bcd-8304-4919-b85f-87d5ca535d2f service neutron] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:19:14.277447 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f61acd7-8554-4f6e-b30c-c2ad3c7ac4ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b07bf0faca47b28b50990d1720853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:19:14.284119 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f61acd7-8554-4f6e-b30c-c2ad3c7ac4ae service nova] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:19:14.284119 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f61acd7-8554-4f6e-b30c-c2ad3c7ac4ae service nova] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:19:14.284119 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f61acd7-8554-4f6e-b30c-c2ad3c7ac4ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:19:14.284119 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-0f61acd7-8554-4f6e-b30c-c2ad3c7ac4ae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:19:14.293018 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f61acd7-8554-4f6e-b30c-c2ad3c7ac4ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:19:14.294651 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f61acd7-8554-4f6e-b30c-c2ad3c7ac4ae service nova] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:19:14.306036 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1273/2568] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:19:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:19:14.420573 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ef3784-5cd6-442e-95d7-94e9744ceb30 service nova] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:19:14.431281 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ef3784-5cd6-442e-95d7-94e9744ceb30 service nova] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:19:14.435045 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9e2166-7ed2-4e54-b4fc-66129fe96aa6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:19:14.435652 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9e2166-7ed2-4e54-b4fc-66129fe96aa6 service neutron] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:19:14.435933 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9e2166-7ed2-4e54-b4fc-66129fe96aa6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:19:14.436580 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9e2166-7ed2-4e54-b4fc-66129fe96aa6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:19:14.438368 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-fc9e2166-7ed2-4e54-b4fc-66129fe96aa6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:19:14.448899 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9e2166-7ed2-4e54-b4fc-66129fe96aa6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:19:14.450384 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9e2166-7ed2-4e54-b4fc-66129fe96aa6 service neutron] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:19:14.459568 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1296/2569] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:19:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:19:15.543078 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f61acd7-8554-4f6e-b30c-c2ad3c7ac4ae service nova] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:19:15.552533 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f61acd7-8554-4f6e-b30c-c2ad3c7ac4ae service nova] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:19:15.556483 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05462f0e-187a-4a08-96dc-a5b4515848ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:19:15.557087 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05462f0e-187a-4a08-96dc-a5b4515848ea service neutron] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:19:15.557368 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05462f0e-187a-4a08-96dc-a5b4515848ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:19:15.557644 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05462f0e-187a-4a08-96dc-a5b4515848ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:19:15.559396 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-05462f0e-187a-4a08-96dc-a5b4515848ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:19:15.573195 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05462f0e-187a-4a08-96dc-a5b4515848ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:19:15.574655 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05462f0e-187a-4a08-96dc-a5b4515848ea service neutron] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:19:15.586414 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1274/2570] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:19:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:19:15.648467 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9e2166-7ed2-4e54-b4fc-66129fe96aa6 service neutron] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:19:15.649869 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-fc9e2166-7ed2-4e54-b4fc-66129fe96aa6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:19:15.719182 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9e2166-7ed2-4e54-b4fc-66129fe96aa6 service neutron] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:19:15.724830 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddbd926-9a47-4aa7-affc-d37626870bd2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '922ef7811aed450baff7f936011b0af1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:19:15.725428 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddbd926-9a47-4aa7-affc-d37626870bd2 service placement] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:19:15.725710 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddbd926-9a47-4aa7-affc-d37626870bd2 service placement] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:19:15.726013 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddbd926-9a47-4aa7-affc-d37626870bd2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:19:15.727756 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-1ddbd926-9a47-4aa7-affc-d37626870bd2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:19:15.739274 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddbd926-9a47-4aa7-affc-d37626870bd2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:19:15.740846 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddbd926-9a47-4aa7-affc-d37626870bd2 service placement] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:19:15.750719 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1297/2571] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:19:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:19:25.580749 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=12 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:19:25.746478 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=38 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:19:39.390030 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02937fe-fde6-4335-8177-66991036e52d None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:19:39.390462 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02937fe-fde6-4335-8177-66991036e52d None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:19:39.390582 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02937fe-fde6-4335-8177-66991036e52d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:19:39.438243 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-e02937fe-fde6-4335-8177-66991036e52d None None] MFA Rules not processed for user `44b37050076b42e6af1a67bd7d6d06e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:19:39.439689 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-e02937fe-fde6-4335-8177-66991036e52d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:19:39.457069 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1275/2572] 10.222.0.39 () {58 vars in 967 bytes} [Fri Oct 14 03:19:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:19:39.498644 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddbd926-9a47-4aa7-affc-d37626870bd2 service placement] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:19:39.519587 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddbd926-9a47-4aa7-affc-d37626870bd2 service placement] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:19:39.524478 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfbe80c-2e64-4592-a218-c39269f773e4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5e07da64804243bc18862072b027a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:19:39.525066 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfbe80c-2e64-4592-a218-c39269f773e4 service swift] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:19:39.525346 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfbe80c-2e64-4592-a218-c39269f773e4 service swift] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:19:39.525618 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfbe80c-2e64-4592-a218-c39269f773e4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:19:39.533513 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfbe80c-2e64-4592-a218-c39269f773e4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:19:39.535112 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfbe80c-2e64-4592-a218-c39269f773e4 service swift] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:19:39.543045 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1298/2573] 10.222.0.39 () {60 vars in 1384 bytes} [Fri Oct 14 03:19:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 14 03:19:47.655110 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02937fe-fde6-4335-8177-66991036e52d None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:19:47.656567 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-e02937fe-fde6-4335-8177-66991036e52d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:19:47.722580 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02937fe-fde6-4335-8177-66991036e52d None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:19:47.726975 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fe8be6-8307-48d1-b9f0-faa921bd3e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:19:47.728275 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fe8be6-8307-48d1-b9f0-faa921bd3e58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:19:47.728600 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fe8be6-8307-48d1-b9f0-faa921bd3e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:19:47.728817 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fe8be6-8307-48d1-b9f0-faa921bd3e58 admin admin] PATH_INFO: `/v3/users/ca9545f301e24e579903df7f87337f8e` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:19:47.729990 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fe8be6-8307-48d1-b9f0-faa921bd3e58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca9545f301e24e579903df7f87337f8e)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:19:47.731919 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fe8be6-8307-48d1-b9f0-faa921bd3e58 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:19:47.924285 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-72fe8be6-8307-48d1-b9f0-faa921bd3e58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca9545f301e24e579903df7f87337f8e'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:19:47.935592 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-72fe8be6-8307-48d1-b9f0-faa921bd3e58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca9545f301e24e579903df7f87337f8e'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:19:47.944198 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-72fe8be6-8307-48d1-b9f0-faa921bd3e58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca9545f301e24e579903df7f87337f8e'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:19:47.944886 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-72fe8be6-8307-48d1-b9f0-faa921bd3e58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca9545f301e24e579903df7f87337f8e'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:19:48.024084 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-72fe8be6-8307-48d1-b9f0-faa921bd3e58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca9545f301e24e579903df7f87337f8e'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:19:48.032016 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1276/2574] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:19:47 2022] DELETE /identity/v3/users/ca9545f301e24e579903df7f87337f8e => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:19:48.259873 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfbe80c-2e64-4592-a218-c39269f773e4 service swift] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:19:48.271808 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfbe80c-2e64-4592-a218-c39269f773e4 service swift] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:19:48.328232 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdf6c3a-92d5-4327-bd5f-5180e4f3233c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:19:48.328855 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdf6c3a-92d5-4327-bd5f-5180e4f3233c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:19:48.329133 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdf6c3a-92d5-4327-bd5f-5180e4f3233c admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:19:48.329405 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdf6c3a-92d5-4327-bd5f-5180e4f3233c admin admin] PATH_INFO: `/v3/projects/f2c0270c3f82423ea62db21e22ac86bd` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:19:48.330553 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdf6c3a-92d5-4327-bd5f-5180e4f3233c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2c0270c3f82423ea62db21e22ac86bd)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:19:48.332446 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdf6c3a-92d5-4327-bd5f-5180e4f3233c admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:19:48.452796 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-2cdf6c3a-92d5-4327-bd5f-5180e4f3233c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2c0270c3f82423ea62db21e22ac86bd', 'request_id': 'req-2cdf6c3a-92d5-4327-bd5f-5180e4f3233c'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:19:48.458804 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-2cdf6c3a-92d5-4327-bd5f-5180e4f3233c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2c0270c3f82423ea62db21e22ac86bd', 'request_id': 'req-2cdf6c3a-92d5-4327-bd5f-5180e4f3233c'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:19:48.539239 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-2cdf6c3a-92d5-4327-bd5f-5180e4f3233c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2c0270c3f82423ea62db21e22ac86bd', 'request_id': 'req-2cdf6c3a-92d5-4327-bd5f-5180e4f3233c'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:19:48.550741 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-2cdf6c3a-92d5-4327-bd5f-5180e4f3233c admin admin] The token cache is being invalidate because project f2c0270c3f82423ea62db21e22ac86bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81517) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:19:48.551033 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-2cdf6c3a-92d5-4327-bd5f-5180e4f3233c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:19:48.551835 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-2cdf6c3a-92d5-4327-bd5f-5180e4f3233c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:19:48.553236 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1299/2575] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:19:48 2022] DELETE /identity/v3/projects/f2c0270c3f82423ea62db21e22ac86bd => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:19:48.557331 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fe8be6-8307-48d1-b9f0-faa921bd3e58 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:19:48.558750 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-72fe8be6-8307-48d1-b9f0-faa921bd3e58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:19:48.571551 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fe8be6-8307-48d1-b9f0-faa921bd3e58 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:19:48.609139 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:19:48.627344 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaa8a6c-c6fb-4147-b28d-9f8cf5b6166b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:19:48.627971 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa8a6c-c6fb-4147-b28d-9f8cf5b6166b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:19:48.628252 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa8a6c-c6fb-4147-b28d-9f8cf5b6166b admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:19:48.628530 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa8a6c-c6fb-4147-b28d-9f8cf5b6166b admin admin] PATH_INFO: `/v3/users/42d9de861bef4f6caaafd28b9333cd14` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:19:48.629708 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaa8a6c-c6fb-4147-b28d-9f8cf5b6166b admin admin] RBAC: Authorizing `identity:delete_user(user_id=42d9de861bef4f6caaafd28b9333cd14)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:19:48.631633 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaa8a6c-c6fb-4147-b28d-9f8cf5b6166b admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:19:48.844522 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-edaa8a6c-c6fb-4147-b28d-9f8cf5b6166b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42d9de861bef4f6caaafd28b9333cd14'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:19:48.851788 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-edaa8a6c-c6fb-4147-b28d-9f8cf5b6166b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42d9de861bef4f6caaafd28b9333cd14'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:19:48.864913 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-edaa8a6c-c6fb-4147-b28d-9f8cf5b6166b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42d9de861bef4f6caaafd28b9333cd14'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:19:48.865614 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-edaa8a6c-c6fb-4147-b28d-9f8cf5b6166b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42d9de861bef4f6caaafd28b9333cd14'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:19:48.963184 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-edaa8a6c-c6fb-4147-b28d-9f8cf5b6166b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42d9de861bef4f6caaafd28b9333cd14'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:19:48.969407 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1277/2576] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:19:48 2022] DELETE /identity/v3/users/42d9de861bef4f6caaafd28b9333cd14 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:19:49.210020 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdf6c3a-92d5-4327-bd5f-5180e4f3233c admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:19:49.217120 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-2cdf6c3a-92d5-4327-bd5f-5180e4f3233c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:19:49.226984 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdf6c3a-92d5-4327-bd5f-5180e4f3233c admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:19:49.285519 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a069f7-9cb6-44e1-8ce7-62e53328a329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:19:49.286205 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a069f7-9cb6-44e1-8ce7-62e53328a329 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:19:49.286458 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a069f7-9cb6-44e1-8ce7-62e53328a329 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:19:49.286733 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a069f7-9cb6-44e1-8ce7-62e53328a329 admin admin] PATH_INFO: `/v3/projects/0155abbda1374196b19097d189a0a1f2` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:19:49.287847 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a069f7-9cb6-44e1-8ce7-62e53328a329 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0155abbda1374196b19097d189a0a1f2)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:19:49.290441 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a069f7-9cb6-44e1-8ce7-62e53328a329 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:19:49.301283 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:19:49.402851 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-07a069f7-9cb6-44e1-8ce7-62e53328a329 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0155abbda1374196b19097d189a0a1f2', 'request_id': 'req-07a069f7-9cb6-44e1-8ce7-62e53328a329'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:19:49.410557 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-07a069f7-9cb6-44e1-8ce7-62e53328a329 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0155abbda1374196b19097d189a0a1f2', 'request_id': 'req-07a069f7-9cb6-44e1-8ce7-62e53328a329'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:19:49.493909 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-07a069f7-9cb6-44e1-8ce7-62e53328a329 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0155abbda1374196b19097d189a0a1f2', 'request_id': 'req-07a069f7-9cb6-44e1-8ce7-62e53328a329'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:19:49.504261 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-07a069f7-9cb6-44e1-8ce7-62e53328a329 admin admin] The token cache is being invalidate because project 0155abbda1374196b19097d189a0a1f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81517) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:19:49.504545 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-07a069f7-9cb6-44e1-8ce7-62e53328a329 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:19:49.505156 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-07a069f7-9cb6-44e1-8ce7-62e53328a329 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:19:49.506754 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1300/2577] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:19:49 2022] DELETE /identity/v3/projects/0155abbda1374196b19097d189a0a1f2 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:19:58.965735 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=35,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:19:59.501137 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=20,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:20:48.763433 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaa8a6c-c6fb-4147-b28d-9f8cf5b6166b admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:20:48.764854 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-edaa8a6c-c6fb-4147-b28d-9f8cf5b6166b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:20:48.832476 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaa8a6c-c6fb-4147-b28d-9f8cf5b6166b admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:20:48.891300 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed9dea3-77b0-4013-8beb-b247aaa4aa6c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '922ef7811aed450baff7f936011b0af1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:20:48.891913 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed9dea3-77b0-4013-8beb-b247aaa4aa6c service placement] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:20:48.892194 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed9dea3-77b0-4013-8beb-b247aaa4aa6c service placement] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:20:48.892467 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed9dea3-77b0-4013-8beb-b247aaa4aa6c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:20:48.894225 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-1ed9dea3-77b0-4013-8beb-b247aaa4aa6c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:20:48.905417 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed9dea3-77b0-4013-8beb-b247aaa4aa6c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:20:48.907019 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed9dea3-77b0-4013-8beb-b247aaa4aa6c service placement] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:20:48.916818 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1278/2578] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:20:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:20:51.752806 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a069f7-9cb6-44e1-8ce7-62e53328a329 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:20:51.754253 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-07a069f7-9cb6-44e1-8ce7-62e53328a329 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:20:51.809203 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a069f7-9cb6-44e1-8ce7-62e53328a329 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:20:51.812643 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a60ce01-01e6-4331-b946-ae37f22ad9f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:20:51.813233 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a60ce01-01e6-4331-b946-ae37f22ad9f1 service neutron] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:20:51.813510 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a60ce01-01e6-4331-b946-ae37f22ad9f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:20:51.813807 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a60ce01-01e6-4331-b946-ae37f22ad9f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:20:51.815461 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-6a60ce01-01e6-4331-b946-ae37f22ad9f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:20:51.872521 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a60ce01-01e6-4331-b946-ae37f22ad9f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:20:51.873907 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a60ce01-01e6-4331-b946-ae37f22ad9f1 service neutron] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:20:51.882677 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1301/2579] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:20:51 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:20:58.912502 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=58 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:20:59.497965 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed9dea3-77b0-4013-8beb-b247aaa4aa6c service placement] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:20:59.507258 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed9dea3-77b0-4013-8beb-b247aaa4aa6c service placement] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:20:59.510827 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cde78af-cf4c-439d-98fa-a61659ef4bdc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:20:59.512044 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cde78af-cf4c-439d-98fa-a61659ef4bdc service neutron] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:20:59.512323 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cde78af-cf4c-439d-98fa-a61659ef4bdc service neutron] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:20:59.512595 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cde78af-cf4c-439d-98fa-a61659ef4bdc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:20:59.514342 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-5cde78af-cf4c-439d-98fa-a61659ef4bdc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:20:59.575424 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cde78af-cf4c-439d-98fa-a61659ef4bdc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:20:59.576853 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cde78af-cf4c-439d-98fa-a61659ef4bdc service neutron] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:20:59.586293 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1279/2580] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:20:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:21:01.878642 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=48 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:21:02.905686 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a60ce01-01e6-4331-b946-ae37f22ad9f1 service neutron] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:21:02.907162 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-6a60ce01-01e6-4331-b946-ae37f22ad9f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:21:02.922563 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a60ce01-01e6-4331-b946-ae37f22ad9f1 service neutron] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:21:02.927608 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac989ef2-f4ed-4fa0-bc51-81e142dc8dc2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '922ef7811aed450baff7f936011b0af1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:21:02.928213 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac989ef2-f4ed-4fa0-bc51-81e142dc8dc2 service placement] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:21:02.928501 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac989ef2-f4ed-4fa0-bc51-81e142dc8dc2 service placement] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:21:02.928779 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac989ef2-f4ed-4fa0-bc51-81e142dc8dc2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:21:02.930525 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-ac989ef2-f4ed-4fa0-bc51-81e142dc8dc2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:21:02.943808 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac989ef2-f4ed-4fa0-bc51-81e142dc8dc2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:21:02.945350 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac989ef2-f4ed-4fa0-bc51-81e142dc8dc2 service placement] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:21:02.955412 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1302/2581] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:21:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:21:06.233850 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cde78af-cf4c-439d-98fa-a61659ef4bdc service neutron] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:21:06.245357 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cde78af-cf4c-439d-98fa-a61659ef4bdc service neutron] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:21:06.250318 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020bd5ee-2f6b-4a36-b6b0-360bc5e92edf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '922ef7811aed450baff7f936011b0af1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:21:06.250910 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020bd5ee-2f6b-4a36-b6b0-360bc5e92edf service placement] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:21:06.251195 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020bd5ee-2f6b-4a36-b6b0-360bc5e92edf service placement] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:21:06.251534 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020bd5ee-2f6b-4a36-b6b0-360bc5e92edf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:21:06.253209 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-020bd5ee-2f6b-4a36-b6b0-360bc5e92edf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:21:06.264849 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020bd5ee-2f6b-4a36-b6b0-360bc5e92edf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:21:06.266476 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020bd5ee-2f6b-4a36-b6b0-360bc5e92edf service placement] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:21:06.277305 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1280/2582] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:21:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:21:06.378735 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac989ef2-f4ed-4fa0-bc51-81e142dc8dc2 service placement] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:21:06.380189 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-ac989ef2-f4ed-4fa0-bc51-81e142dc8dc2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:21:06.395502 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac989ef2-f4ed-4fa0-bc51-81e142dc8dc2 service placement] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:21:06.400075 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851e6353-e447-4da7-a62d-e116edd8271e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:21:06.400700 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851e6353-e447-4da7-a62d-e116edd8271e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:21:06.400980 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851e6353-e447-4da7-a62d-e116edd8271e admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:21:06.401262 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851e6353-e447-4da7-a62d-e116edd8271e admin admin] PATH_INFO: `/v3/users/42efd916cb4b4980a45199d8cc9a4389` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:21:06.402488 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851e6353-e447-4da7-a62d-e116edd8271e admin admin] RBAC: Authorizing `identity:delete_user(user_id=42efd916cb4b4980a45199d8cc9a4389)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:21:06.404461 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851e6353-e447-4da7-a62d-e116edd8271e admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:21:06.556995 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-851e6353-e447-4da7-a62d-e116edd8271e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42efd916cb4b4980a45199d8cc9a4389'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:21:06.557653 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-851e6353-e447-4da7-a62d-e116edd8271e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42efd916cb4b4980a45199d8cc9a4389'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:21:06.640495 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-851e6353-e447-4da7-a62d-e116edd8271e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42efd916cb4b4980a45199d8cc9a4389'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:21:06.641136 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-851e6353-e447-4da7-a62d-e116edd8271e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42efd916cb4b4980a45199d8cc9a4389'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:21:06.648370 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-851e6353-e447-4da7-a62d-e116edd8271e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42efd916cb4b4980a45199d8cc9a4389'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:21:06.665502 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1303/2583] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:21:06 2022] DELETE /identity/v3/users/42efd916cb4b4980a45199d8cc9a4389 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:21:06.916516 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020bd5ee-2f6b-4a36-b6b0-360bc5e92edf service placement] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:21:06.928858 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020bd5ee-2f6b-4a36-b6b0-360bc5e92edf service placement] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:21:06.984428 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1265a6-9f6a-42bf-a78a-ea8f2aa2c983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:21:06.985059 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1265a6-9f6a-42bf-a78a-ea8f2aa2c983 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:21:06.985341 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1265a6-9f6a-42bf-a78a-ea8f2aa2c983 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:21:06.985621 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1265a6-9f6a-42bf-a78a-ea8f2aa2c983 admin admin] PATH_INFO: `/v3/projects/0a302ca039794ab599996fcda417a1c6` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:21:06.986779 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1265a6-9f6a-42bf-a78a-ea8f2aa2c983 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a302ca039794ab599996fcda417a1c6)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:21:06.988665 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1265a6-9f6a-42bf-a78a-ea8f2aa2c983 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:21:07.109222 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-6c1265a6-9f6a-42bf-a78a-ea8f2aa2c983 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a302ca039794ab599996fcda417a1c6', 'request_id': 'req-6c1265a6-9f6a-42bf-a78a-ea8f2aa2c983'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:21:07.115644 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-6c1265a6-9f6a-42bf-a78a-ea8f2aa2c983 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a302ca039794ab599996fcda417a1c6', 'request_id': 'req-6c1265a6-9f6a-42bf-a78a-ea8f2aa2c983'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:21:07.194982 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-6c1265a6-9f6a-42bf-a78a-ea8f2aa2c983 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a302ca039794ab599996fcda417a1c6', 'request_id': 'req-6c1265a6-9f6a-42bf-a78a-ea8f2aa2c983'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:21:07.206258 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-6c1265a6-9f6a-42bf-a78a-ea8f2aa2c983 admin admin] The token cache is being invalidate because project 0a302ca039794ab599996fcda417a1c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:21:07.206551 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-6c1265a6-9f6a-42bf-a78a-ea8f2aa2c983 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:21:07.207173 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-6c1265a6-9f6a-42bf-a78a-ea8f2aa2c983 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:21:07.208652 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1281/2584] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:21:06 2022] DELETE /identity/v3/projects/0a302ca039794ab599996fcda417a1c6 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:21:07.212714 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851e6353-e447-4da7-a62d-e116edd8271e admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:21:07.214166 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-851e6353-e447-4da7-a62d-e116edd8271e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:21:07.227892 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851e6353-e447-4da7-a62d-e116edd8271e admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:21:07.283538 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b5c60a-2fb6-4129-8a9a-97a124b3c741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:21:07.284160 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b5c60a-2fb6-4129-8a9a-97a124b3c741 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:21:07.284445 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b5c60a-2fb6-4129-8a9a-97a124b3c741 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:21:07.284740 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b5c60a-2fb6-4129-8a9a-97a124b3c741 admin admin] PATH_INFO: `/v3/users/4b0a436168bb45aa9635df4774e275dc` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:21:07.285906 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b5c60a-2fb6-4129-8a9a-97a124b3c741 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b0a436168bb45aa9635df4774e275dc)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:21:07.287829 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b5c60a-2fb6-4129-8a9a-97a124b3c741 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:21:07.431665 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-88b5c60a-2fb6-4129-8a9a-97a124b3c741 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b0a436168bb45aa9635df4774e275dc'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:21:07.432324 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-88b5c60a-2fb6-4129-8a9a-97a124b3c741 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b0a436168bb45aa9635df4774e275dc'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:21:07.492360 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-88b5c60a-2fb6-4129-8a9a-97a124b3c741 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b0a436168bb45aa9635df4774e275dc'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:21:07.492907 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-88b5c60a-2fb6-4129-8a9a-97a124b3c741 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b0a436168bb45aa9635df4774e275dc'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:21:07.498441 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:21:07.500323 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-88b5c60a-2fb6-4129-8a9a-97a124b3c741 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b0a436168bb45aa9635df4774e275dc'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:21:07.518151 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1304/2585] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:21:07 2022] DELETE /identity/v3/users/4b0a436168bb45aa9635df4774e275dc => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 03:21:07.979223 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1265a6-9f6a-42bf-a78a-ea8f2aa2c983 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:21:07.981331 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-6c1265a6-9f6a-42bf-a78a-ea8f2aa2c983 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:21:07.994717 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1265a6-9f6a-42bf-a78a-ea8f2aa2c983 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:21:08.005840 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:21:08.055988 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58342417-1982-443c-9631-c52a5956b8a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:21:08.056617 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58342417-1982-443c-9631-c52a5956b8a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:21:08.056896 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58342417-1982-443c-9631-c52a5956b8a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:21:08.057168 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58342417-1982-443c-9631-c52a5956b8a7 admin admin] PATH_INFO: `/v3/projects/cedd385212eb4fb2a79a18020ecce76b` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:21:08.058284 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58342417-1982-443c-9631-c52a5956b8a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cedd385212eb4fb2a79a18020ecce76b)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:21:08.060782 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58342417-1982-443c-9631-c52a5956b8a7 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:21:08.160309 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-58342417-1982-443c-9631-c52a5956b8a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cedd385212eb4fb2a79a18020ecce76b', 'request_id': 'req-58342417-1982-443c-9631-c52a5956b8a7'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:21:08.166482 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-58342417-1982-443c-9631-c52a5956b8a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cedd385212eb4fb2a79a18020ecce76b', 'request_id': 'req-58342417-1982-443c-9631-c52a5956b8a7'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:21:08.256974 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-58342417-1982-443c-9631-c52a5956b8a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cedd385212eb4fb2a79a18020ecce76b', 'request_id': 'req-58342417-1982-443c-9631-c52a5956b8a7'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:21:08.267779 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-58342417-1982-443c-9631-c52a5956b8a7 admin admin] The token cache is being invalidate because project cedd385212eb4fb2a79a18020ecce76b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:21:08.268067 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-58342417-1982-443c-9631-c52a5956b8a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:21:08.268694 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-58342417-1982-443c-9631-c52a5956b8a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:21:08.270207 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1282/2586] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:21:07 2022] DELETE /identity/v3/projects/cedd385212eb4fb2a79a18020ecce76b => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:21:08.275337 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667d846f-dfb8-4073-9444-e08a135643f8 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:21:08.275649 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667d846f-dfb8-4073-9444-e08a135643f8 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:21:08.275936 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667d846f-dfb8-4073-9444-e08a135643f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:21:08.296117 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-667d846f-dfb8-4073-9444-e08a135643f8 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:21:08.297305 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-667d846f-dfb8-4073-9444-e08a135643f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:21:08.354700 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1305/2587] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:21:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:21:08.363225 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e32131-7c9c-410c-a0f2-f215a000cbd2 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:21:08.363519 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e32131-7c9c-410c-a0f2-f215a000cbd2 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:21:08.363792 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e32131-7c9c-410c-a0f2-f215a000cbd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:21:08.383210 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-a7e32131-7c9c-410c-a0f2-f215a000cbd2 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:21:08.384398 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-a7e32131-7c9c-410c-a0f2-f215a000cbd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:21:08.392198 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1283/2588] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:21:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:21:08.396324 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667d846f-dfb8-4073-9444-e08a135643f8 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:21:08.406801 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667d846f-dfb8-4073-9444-e08a135643f8 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:21:08.411388 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4e5afa-7ae8-46d4-afdd-34c3aad5c72b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:21:08.411981 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4e5afa-7ae8-46d4-afdd-34c3aad5c72b admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:21:08.412265 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4e5afa-7ae8-46d4-afdd-34c3aad5c72b admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:21:08.412540 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4e5afa-7ae8-46d4-afdd-34c3aad5c72b admin admin] PATH_INFO: `/v3/domains` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:21:08.413212 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4e5afa-7ae8-46d4-afdd-34c3aad5c72b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81517) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 14 03:21:08.413521 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4e5afa-7ae8-46d4-afdd-34c3aad5c72b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:21:08.415229 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4e5afa-7ae8-46d4-afdd-34c3aad5c72b admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:21:08.427357 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1306/2589] 10.222.0.39 () {58 vars in 1153 bytes} [Fri Oct 14 03:21:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 14 03:21:08.431524 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e32131-7c9c-410c-a0f2-f215a000cbd2 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:21:08.440601 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e32131-7c9c-410c-a0f2-f215a000cbd2 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:21:08.445417 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855cb68c-9b27-48b8-b4eb-d4fe0365087a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:21:08.446049 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855cb68c-9b27-48b8-b4eb-d4fe0365087a admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:21:08.446333 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855cb68c-9b27-48b8-b4eb-d4fe0365087a admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:21:08.446606 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855cb68c-9b27-48b8-b4eb-d4fe0365087a admin admin] PATH_INFO: `/v3/projects` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:21:08.447280 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855cb68c-9b27-48b8-b4eb-d4fe0365087a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:21:08.449172 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855cb68c-9b27-48b8-b4eb-d4fe0365087a admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:21:08.545287 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1284/2590] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:21:08 2022] POST /identity/v3/projects => generated 379 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 03:21:08.549293 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4e5afa-7ae8-46d4-afdd-34c3aad5c72b admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:21:08.559508 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4e5afa-7ae8-46d4-afdd-34c3aad5c72b admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:21:08.614310 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11905949-ac9c-4f7f-a3ea-20cf8b368db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:21:08.615153 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11905949-ac9c-4f7f-a3ea-20cf8b368db9 admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:21:08.615434 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11905949-ac9c-4f7f-a3ea-20cf8b368db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:21:08.615706 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11905949-ac9c-4f7f-a3ea-20cf8b368db9 admin admin] PATH_INFO: `/v3/users` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:21:08.616329 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11905949-ac9c-4f7f-a3ea-20cf8b368db9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:21:08.618263 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11905949-ac9c-4f7f-a3ea-20cf8b368db9 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:21:08.703996 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1307/2591] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:21:08 2022] POST /identity/v3/users => generated 290 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 03:21:08.708107 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855cb68c-9b27-48b8-b4eb-d4fe0365087a admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:21:08.718945 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855cb68c-9b27-48b8-b4eb-d4fe0365087a admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:21:08.723506 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6d9778-dc8a-411b-b19f-77cd324f040b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:21:08.724096 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6d9778-dc8a-411b-b19f-77cd324f040b admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:21:08.724380 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6d9778-dc8a-411b-b19f-77cd324f040b admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:21:08.724655 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6d9778-dc8a-411b-b19f-77cd324f040b admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:21:08.725350 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6d9778-dc8a-411b-b19f-77cd324f040b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:21:08.726987 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6d9778-dc8a-411b-b19f-77cd324f040b admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:21:08.736665 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1285/2592] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:21:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:21:08.740739 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11905949-ac9c-4f7f-a3ea-20cf8b368db9 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:21:08.750620 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11905949-ac9c-4f7f-a3ea-20cf8b368db9 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:21:08.755131 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838cd22b-bfb5-40fd-9896-7e543be6c742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:21:08.755714 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838cd22b-bfb5-40fd-9896-7e543be6c742 admin admin] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:21:08.755993 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838cd22b-bfb5-40fd-9896-7e543be6c742 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:21:08.756262 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838cd22b-bfb5-40fd-9896-7e543be6c742 admin admin] PATH_INFO: `/v3/roles` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:21:08.756891 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838cd22b-bfb5-40fd-9896-7e543be6c742 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:21:08.758506 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838cd22b-bfb5-40fd-9896-7e543be6c742 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:21:08.769264 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1308/2593] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:21:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:21:08.773168 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6d9778-dc8a-411b-b19f-77cd324f040b admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:21:08.783334 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6d9778-dc8a-411b-b19f-77cd324f040b admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:21:08.787796 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581aa0f1-ed00-434c-a1bd-c35816541f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:21:08.788427 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581aa0f1-ed00-434c-a1bd-c35816541f4d admin admin] REQUEST_METHOD: `PUT` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:21:08.788706 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581aa0f1-ed00-434c-a1bd-c35816541f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:21:08.788982 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581aa0f1-ed00-434c-a1bd-c35816541f4d admin admin] PATH_INFO: `/v3/projects/094ec37b955e4518b7abffd09afef4f0/users/fcc0e4c5bee64180884513f13757140c/roles/9bd0470b544240209a420f640c79c06c` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:21:08.804646 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581aa0f1-ed00-434c-a1bd-c35816541f4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=094ec37b955e4518b7abffd09afef4f0, user_id=fcc0e4c5bee64180884513f13757140c, role_id=9bd0470b544240209a420f640c79c06c)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:21:08.807248 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581aa0f1-ed00-434c-a1bd-c35816541f4d admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:21:08.892760 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1286/2594] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:21:08 2022] PUT /identity/v3/projects/094ec37b955e4518b7abffd09afef4f0/users/fcc0e4c5bee64180884513f13757140c/roles/9bd0470b544240209a420f640c79c06c => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:21:08.897764 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32375f2-e830-4a1d-8baa-65e1ee4af5c0 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:21:08.898058 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32375f2-e830-4a1d-8baa-65e1ee4af5c0 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:21:08.898353 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32375f2-e830-4a1d-8baa-65e1ee4af5c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:21:08.917638 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-c32375f2-e830-4a1d-8baa-65e1ee4af5c0 None None] MFA Rules not processed for user `316b39adbf9f4b0d9c13dc4c11a24e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:21:08.918897 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-c32375f2-e830-4a1d-8baa-65e1ee4af5c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:21:08.958194 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:21:08.976946 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1309/2595] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:21:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:21:08.986696 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581aa0f1-ed00-434c-a1bd-c35816541f4d admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:21:08.988393 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-581aa0f1-ed00-434c-a1bd-c35816541f4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:21:09.033004 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:21:09.048040 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581aa0f1-ed00-434c-a1bd-c35816541f4d admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:21:09.051721 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2118c901-9ddd-4794-9504-d2cc896e4171 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:21:09.053364 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2118c901-9ddd-4794-9504-d2cc896e4171 service neutron] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:21:09.053364 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2118c901-9ddd-4794-9504-d2cc896e4171 service neutron] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:21:09.053364 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2118c901-9ddd-4794-9504-d2cc896e4171 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:21:09.062441 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2118c901-9ddd-4794-9504-d2cc896e4171 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:21:09.063824 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2118c901-9ddd-4794-9504-d2cc896e4171 service neutron] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:21:09.073140 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1287/2596] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:21:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:21:14.074589 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f105ded2-a52c-4d89-869a-f0507aeada71 None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:21:14.075454 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f105ded2-a52c-4d89-869a-f0507aeada71 None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:21:14.076079 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f105ded2-a52c-4d89-869a-f0507aeada71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:21:14.115944 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-f105ded2-a52c-4d89-869a-f0507aeada71 None None] MFA Rules not processed for user `fcc0e4c5bee64180884513f13757140c`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:21:14.117936 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-f105ded2-a52c-4d89-869a-f0507aeada71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:21:14.198741 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1310/2597] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:21:14 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:21:14.202967 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2118c901-9ddd-4794-9504-d2cc896e4171 service neutron] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:21:14.215428 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2118c901-9ddd-4794-9504-d2cc896e4171 service neutron] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:21:14.220717 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcddbb2-a9a0-4aaf-9f80-7ee6d762f532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:21:14.221413 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcddbb2-a9a0-4aaf-9f80-7ee6d762f532 admin admin] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:21:14.221943 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcddbb2-a9a0-4aaf-9f80-7ee6d762f532 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:21:14.222224 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcddbb2-a9a0-4aaf-9f80-7ee6d762f532 admin admin] PATH_INFO: `/v3/projects` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:21:14.223826 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcddbb2-a9a0-4aaf-9f80-7ee6d762f532 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:21:14.226485 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcddbb2-a9a0-4aaf-9f80-7ee6d762f532 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:21:14.335478 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1288/2598] 10.222.0.39 () {60 vars in 1153 bytes} [Fri Oct 14 03:21:14 2022] POST /identity/v3/projects => generated 377 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 14 03:21:14.339696 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f105ded2-a52c-4d89-869a-f0507aeada71 None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:21:14.351277 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f105ded2-a52c-4d89-869a-f0507aeada71 None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:21:14.416386 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a214396b-ee42-4bb0-acda-ba153d62e5cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:21:14.417024 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a214396b-ee42-4bb0-acda-ba153d62e5cf admin admin] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:21:14.417289 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a214396b-ee42-4bb0-acda-ba153d62e5cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:21:14.417564 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a214396b-ee42-4bb0-acda-ba153d62e5cf admin admin] PATH_INFO: `/v3/users` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:21:14.418398 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a214396b-ee42-4bb0-acda-ba153d62e5cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:21:14.420330 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a214396b-ee42-4bb0-acda-ba153d62e5cf admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:21:14.521828 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1311/2599] 10.222.0.39 () {60 vars in 1144 bytes} [Fri Oct 14 03:21:14 2022] POST /identity/v3/users => generated 295 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 14 03:21:14.526166 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcddbb2-a9a0-4aaf-9f80-7ee6d762f532 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:21:14.536620 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcddbb2-a9a0-4aaf-9f80-7ee6d762f532 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:21:14.542000 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6576971c-0075-40ad-a35c-039712ac5456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:21:14.543777 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6576971c-0075-40ad-a35c-039712ac5456 admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:21:14.543777 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6576971c-0075-40ad-a35c-039712ac5456 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:21:14.543777 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6576971c-0075-40ad-a35c-039712ac5456 admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:21:14.545091 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6576971c-0075-40ad-a35c-039712ac5456 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:21:14.545817 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6576971c-0075-40ad-a35c-039712ac5456 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:21:14.556522 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1289/2600] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:21:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 14 03:21:14.561344 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a214396b-ee42-4bb0-acda-ba153d62e5cf admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:21:14.571474 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a214396b-ee42-4bb0-acda-ba153d62e5cf admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:21:14.576820 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3fef76-6102-4bdb-8f10-95e14d6c7263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:21:14.577469 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3fef76-6102-4bdb-8f10-95e14d6c7263 admin admin] REQUEST_METHOD: `PUT` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:21:14.577768 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3fef76-6102-4bdb-8f10-95e14d6c7263 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:21:14.578070 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3fef76-6102-4bdb-8f10-95e14d6c7263 admin admin] PATH_INFO: `/v3/projects/ce079564be954618a73a9c64c4f551d5/users/f58f655bc33b4577bfaa3b6aa00882ba/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:21:14.592571 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3fef76-6102-4bdb-8f10-95e14d6c7263 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce079564be954618a73a9c64c4f551d5, user_id=f58f655bc33b4577bfaa3b6aa00882ba, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:21:14.595283 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3fef76-6102-4bdb-8f10-95e14d6c7263 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:21:14.646989 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1312/2601] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:21:14 2022] PUT /identity/v3/projects/ce079564be954618a73a9c64c4f551d5/users/f58f655bc33b4577bfaa3b6aa00882ba/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:21:14.650914 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6576971c-0075-40ad-a35c-039712ac5456 admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:21:14.660958 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6576971c-0075-40ad-a35c-039712ac5456 admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:21:14.716642 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646e7155-3dce-40d5-a4ca-6d191f99932f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:21:14.717238 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646e7155-3dce-40d5-a4ca-6d191f99932f admin admin] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:21:14.717515 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646e7155-3dce-40d5-a4ca-6d191f99932f admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:21:14.717809 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646e7155-3dce-40d5-a4ca-6d191f99932f admin admin] PATH_INFO: `/v3/roles` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:21:14.718451 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646e7155-3dce-40d5-a4ca-6d191f99932f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:21:14.720028 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646e7155-3dce-40d5-a4ca-6d191f99932f admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:21:14.729439 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1290/2602] 10.222.0.39 () {58 vars in 1122 bytes} [Fri Oct 14 03:21:14 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 14 03:21:14.734142 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3fef76-6102-4bdb-8f10-95e14d6c7263 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:21:14.744817 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3fef76-6102-4bdb-8f10-95e14d6c7263 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:21:14.750115 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdb4deb-937a-4dba-a6fd-2d79b01503d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:21:14.750763 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb4deb-937a-4dba-a6fd-2d79b01503d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:21:14.751049 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb4deb-937a-4dba-a6fd-2d79b01503d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:21:14.751326 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb4deb-937a-4dba-a6fd-2d79b01503d3 admin admin] PATH_INFO: `/v3/projects/ce079564be954618a73a9c64c4f551d5/users/f58f655bc33b4577bfaa3b6aa00882ba/roles/9e86d182c0594e0785b72ba5f390a114` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:21:14.753298 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdb4deb-937a-4dba-a6fd-2d79b01503d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce079564be954618a73a9c64c4f551d5, user_id=f58f655bc33b4577bfaa3b6aa00882ba, role_id=9e86d182c0594e0785b72ba5f390a114)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:21:14.756010 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdb4deb-937a-4dba-a6fd-2d79b01503d3 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:21:14.768175 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1313/2603] 10.222.0.39 () {60 vars in 1483 bytes} [Fri Oct 14 03:21:14 2022] PUT /identity/v3/projects/ce079564be954618a73a9c64c4f551d5/users/f58f655bc33b4577bfaa3b6aa00882ba/roles/9e86d182c0594e0785b72ba5f390a114 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:21:20.245615 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f06bdf1-fbb7-4109-b2bf-e9d71bf75302 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:21:20.246706 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f06bdf1-fbb7-4109-b2bf-e9d71bf75302 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:21:20.247066 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f06bdf1-fbb7-4109-b2bf-e9d71bf75302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:21:20.276540 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-6f06bdf1-fbb7-4109-b2bf-e9d71bf75302 None None] MFA Rules not processed for user `f58f655bc33b4577bfaa3b6aa00882ba`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:21:20.278408 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-6f06bdf1-fbb7-4109-b2bf-e9d71bf75302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:21:20.348515 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1291/2604] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:21:20 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:21:20.364068 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdb4deb-937a-4dba-a6fd-2d79b01503d3 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:21:20.366223 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-3cdb4deb-937a-4dba-a6fd-2d79b01503d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:21:20.400496 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:21:20.453033 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdb4deb-937a-4dba-a6fd-2d79b01503d3 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:21:20.459147 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61396edb-f4f6-4d06-8526-2b1335e4ece9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b07bf0faca47b28b50990d1720853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:21:20.459761 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61396edb-f4f6-4d06-8526-2b1335e4ece9 service nova] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:21:20.460043 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61396edb-f4f6-4d06-8526-2b1335e4ece9 service nova] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:21:20.460388 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61396edb-f4f6-4d06-8526-2b1335e4ece9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:21:20.519374 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61396edb-f4f6-4d06-8526-2b1335e4ece9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:21:20.520989 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61396edb-f4f6-4d06-8526-2b1335e4ece9 service nova] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:21:20.531823 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1314/2605] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:21:20 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:21:30.345522 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=86,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:21:30.720273 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=44 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:21:40.661223 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30111872-0f50-4ec3-816d-44015a57c152 None None] REQUEST_METHOD: `POST` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:21:40.661752 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30111872-0f50-4ec3-816d-44015a57c152 None None] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:21:40.661752 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30111872-0f50-4ec3-816d-44015a57c152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:21:40.681292 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.auth.core [None req-30111872-0f50-4ec3-816d-44015a57c152 None None] MFA Rules not processed for user `fcc0e4c5bee64180884513f13757140c`. Rule list: `[]` (Enabled: `True`). {{(pid=81518) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:21:40.682540 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-30111872-0f50-4ec3-816d-44015a57c152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:21:40.689404 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1292/2606] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:21:40 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:21:41.012135 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61396edb-f4f6-4d06-8526-2b1335e4ece9 service nova] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:21:41.065856 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61396edb-f4f6-4d06-8526-2b1335e4ece9 service nova] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:21:41.069450 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc048dce-58b4-4dbd-835b-07bbb971215b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:21:41.070010 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc048dce-58b4-4dbd-835b-07bbb971215b service neutron] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:21:41.070294 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc048dce-58b4-4dbd-835b-07bbb971215b service neutron] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:21:41.070560 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc048dce-58b4-4dbd-835b-07bbb971215b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:21:41.072262 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-fc048dce-58b4-4dbd-835b-07bbb971215b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:21:41.082110 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc048dce-58b4-4dbd-835b-07bbb971215b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:21:41.083525 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc048dce-58b4-4dbd-835b-07bbb971215b service neutron] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:21:41.092008 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1315/2607] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:21:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:21:50.675649 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=3 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:21:51.087768 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=26 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:21:59.516645 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30111872-0f50-4ec3-816d-44015a57c152 None None] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:21:59.526459 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30111872-0f50-4ec3-816d-44015a57c152 None None] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:21:59.530217 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade4716e-d06b-4fec-b42f-0f7ca3809c3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:21:59.530812 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade4716e-d06b-4fec-b42f-0f7ca3809c3d service neutron] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:21:59.531092 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade4716e-d06b-4fec-b42f-0f7ca3809c3d service neutron] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:21:59.531366 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade4716e-d06b-4fec-b42f-0f7ca3809c3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:21:59.540319 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade4716e-d06b-4fec-b42f-0f7ca3809c3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:21:59.541890 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade4716e-d06b-4fec-b42f-0f7ca3809c3d service neutron] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:21:59.551322 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1293/2608] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:21:59 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:22:09.547014 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=6 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:23:40.321299 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc048dce-58b4-4dbd-835b-07bbb971215b service neutron] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:23:40.322712 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-fc048dce-58b4-4dbd-835b-07bbb971215b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:23:40.333463 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc048dce-58b4-4dbd-835b-07bbb971215b service neutron] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:23:40.337087 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b186ea-bd99-401d-a1c8-0c957923bd42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:23:40.337669 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b186ea-bd99-401d-a1c8-0c957923bd42 service neutron] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:23:40.337976 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b186ea-bd99-401d-a1c8-0c957923bd42 service neutron] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:23:40.338245 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b186ea-bd99-401d-a1c8-0c957923bd42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:23:40.339881 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-a7b186ea-bd99-401d-a1c8-0c957923bd42 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:23:40.349862 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b186ea-bd99-401d-a1c8-0c957923bd42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:23:40.351462 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b186ea-bd99-401d-a1c8-0c957923bd42 service neutron] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:23:40.359867 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1316/2609] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:23:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:23:50.355986 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=6 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:23:58.896309 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade4716e-d06b-4fec-b42f-0f7ca3809c3d service neutron] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:23:58.905044 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade4716e-d06b-4fec-b42f-0f7ca3809c3d service neutron] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:23:58.908632 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51824e65-3491-402e-8c3b-d85324cb4d7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:23:58.909217 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51824e65-3491-402e-8c3b-d85324cb4d7f service neutron] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:23:58.909495 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51824e65-3491-402e-8c3b-d85324cb4d7f service neutron] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:23:58.909863 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51824e65-3491-402e-8c3b-d85324cb4d7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:23:58.911619 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-51824e65-3491-402e-8c3b-d85324cb4d7f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:23:58.925820 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51824e65-3491-402e-8c3b-d85324cb4d7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:23:58.927276 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51824e65-3491-402e-8c3b-d85324cb4d7f service neutron] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:23:58.938739 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1294/2610] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:23:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:24:00.119300 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b186ea-bd99-401d-a1c8-0c957923bd42 service neutron] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:24:00.128828 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b186ea-bd99-401d-a1c8-0c957923bd42 service neutron] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:24:00.132451 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c017721-1a4a-4dd1-9288-af959f9691dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:24:00.133045 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c017721-1a4a-4dd1-9288-af959f9691dc service neutron] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:24:00.133325 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c017721-1a4a-4dd1-9288-af959f9691dc service neutron] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:24:00.133598 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c017721-1a4a-4dd1-9288-af959f9691dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:24:00.135358 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-2c017721-1a4a-4dd1-9288-af959f9691dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:24:00.148685 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c017721-1a4a-4dd1-9288-af959f9691dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:24:00.150157 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c017721-1a4a-4dd1-9288-af959f9691dc service neutron] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:24:00.161161 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1317/2611] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:24:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:24:01.840649 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51824e65-3491-402e-8c3b-d85324cb4d7f service neutron] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:24:01.851904 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51824e65-3491-402e-8c3b-d85324cb4d7f service neutron] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:24:01.857752 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cd6be2-9b0e-4093-9416-494d87e0099a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b07bf0faca47b28b50990d1720853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:24:01.858381 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd6be2-9b0e-4093-9416-494d87e0099a service nova] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:24:01.858662 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd6be2-9b0e-4093-9416-494d87e0099a service nova] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:24:01.858939 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd6be2-9b0e-4093-9416-494d87e0099a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:24:01.860743 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-28cd6be2-9b0e-4093-9416-494d87e0099a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:24:01.874013 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cd6be2-9b0e-4093-9416-494d87e0099a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:24:01.875627 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cd6be2-9b0e-4093-9416-494d87e0099a service nova] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:24:01.889985 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1295/2612] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:24:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:24:03.608855 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a13e381-7c78-4f1c-8a06-3cd4e49cd18a None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:24:03.609327 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a13e381-7c78-4f1c-8a06-3cd4e49cd18a None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:24:03.609432 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a13e381-7c78-4f1c-8a06-3cd4e49cd18a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:24:03.638264 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-5a13e381-7c78-4f1c-8a06-3cd4e49cd18a None None] MFA Rules not processed for user `fcc0e4c5bee64180884513f13757140c`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:24:03.639450 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-5a13e381-7c78-4f1c-8a06-3cd4e49cd18a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:24:03.663090 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1318/2613] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:24:03 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:24:11.885521 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=12 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:24:13.661317 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=21 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:24:23.773472 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cd6be2-9b0e-4093-9416-494d87e0099a service nova] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:24:23.814103 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cd6be2-9b0e-4093-9416-494d87e0099a service nova] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:24:23.820044 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271eeb87-ac0c-408b-99cd-d26b14efda3f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b07bf0faca47b28b50990d1720853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:24:23.820594 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271eeb87-ac0c-408b-99cd-d26b14efda3f service nova] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:24:23.820881 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271eeb87-ac0c-408b-99cd-d26b14efda3f service nova] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:24:23.821174 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271eeb87-ac0c-408b-99cd-d26b14efda3f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:24:23.822971 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-271eeb87-ac0c-408b-99cd-d26b14efda3f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:24:23.846047 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271eeb87-ac0c-408b-99cd-d26b14efda3f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:24:23.847657 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271eeb87-ac0c-408b-99cd-d26b14efda3f service nova] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:24:23.858443 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1296/2614] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:24:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:24:28.730374 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a13e381-7c78-4f1c-8a06-3cd4e49cd18a None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:24:28.731797 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-5a13e381-7c78-4f1c-8a06-3cd4e49cd18a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:24:28.748436 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a13e381-7c78-4f1c-8a06-3cd4e49cd18a None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:24:28.754926 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c834f84-a6a7-48dc-8574-0c3f0fad3008 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b07bf0faca47b28b50990d1720853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:24:28.755833 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c834f84-a6a7-48dc-8574-0c3f0fad3008 service nova] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:24:28.756113 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c834f84-a6a7-48dc-8574-0c3f0fad3008 service nova] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:24:28.756390 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c834f84-a6a7-48dc-8574-0c3f0fad3008 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:24:28.758193 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-1c834f84-a6a7-48dc-8574-0c3f0fad3008 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:24:28.770947 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c834f84-a6a7-48dc-8574-0c3f0fad3008 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:24:28.772560 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c834f84-a6a7-48dc-8574-0c3f0fad3008 service nova] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:24:28.783237 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1319/2615] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:24:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:24:33.854543 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=24 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:24:38.778683 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=6 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:25:50.764479 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271eeb87-ac0c-408b-99cd-d26b14efda3f service nova] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:25:50.765993 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-271eeb87-ac0c-408b-99cd-d26b14efda3f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:25:50.853750 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271eeb87-ac0c-408b-99cd-d26b14efda3f service nova] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:25:50.858759 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c88a199-3bd8-4421-a6b6-99322bb1bff4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '922ef7811aed450baff7f936011b0af1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:25:50.859355 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c88a199-3bd8-4421-a6b6-99322bb1bff4 service placement] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:25:50.859632 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c88a199-3bd8-4421-a6b6-99322bb1bff4 service placement] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:25:50.859923 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c88a199-3bd8-4421-a6b6-99322bb1bff4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:25:50.861623 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-1c88a199-3bd8-4421-a6b6-99322bb1bff4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:25:50.873748 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c88a199-3bd8-4421-a6b6-99322bb1bff4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:25:50.875355 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c88a199-3bd8-4421-a6b6-99322bb1bff4 service placement] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:25:50.885775 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1297/2616] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:25:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 14 03:26:00.881325 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=41 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:26:04.664079 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ec6f93-0d24-4319-ad79-9cf4a799cc4c None None] REQUEST_METHOD: `POST` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:26:04.664579 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ec6f93-0d24-4319-ad79-9cf4a799cc4c None None] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:26:04.664706 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ec6f93-0d24-4319-ad79-9cf4a799cc4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:26:04.684084 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.auth.core [None req-e4ec6f93-0d24-4319-ad79-9cf4a799cc4c None None] MFA Rules not processed for user `fcc0e4c5bee64180884513f13757140c`. Rule list: `[]` (Enabled: `True`). {{(pid=81517) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 14 03:26:04.685283 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-e4ec6f93-0d24-4319-ad79-9cf4a799cc4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:26:04.692201 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1320/2617] 10.222.0.39 () {58 vars in 958 bytes} [Fri Oct 14 03:26:04 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 14 03:26:05.010706 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c88a199-3bd8-4421-a6b6-99322bb1bff4 service placement] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:26:05.029959 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c88a199-3bd8-4421-a6b6-99322bb1bff4 service placement] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:26:05.033549 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123dd77b-c3c2-4fa8-8d2a-0f8d57819c46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:26:05.034181 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123dd77b-c3c2-4fa8-8d2a-0f8d57819c46 service neutron] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:26:05.034463 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123dd77b-c3c2-4fa8-8d2a-0f8d57819c46 service neutron] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:26:05.034739 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123dd77b-c3c2-4fa8-8d2a-0f8d57819c46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:26:05.036452 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-123dd77b-c3c2-4fa8-8d2a-0f8d57819c46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:26:05.057997 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123dd77b-c3c2-4fa8-8d2a-0f8d57819c46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:26:05.059413 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123dd77b-c3c2-4fa8-8d2a-0f8d57819c46 service neutron] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:26:05.067926 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1298/2618] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:26:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:26:05.770519 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ec6f93-0d24-4319-ad79-9cf4a799cc4c None None] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:26:05.771957 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-e4ec6f93-0d24-4319-ad79-9cf4a799cc4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:26:05.787974 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ec6f93-0d24-4319-ad79-9cf4a799cc4c None None] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:26:05.793423 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81af8a84-5964-4621-8375-aea0014b2cea service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '922ef7811aed450baff7f936011b0af1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:26:05.794100 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81af8a84-5964-4621-8375-aea0014b2cea service placement] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:26:05.794384 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81af8a84-5964-4621-8375-aea0014b2cea service placement] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:26:05.794684 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81af8a84-5964-4621-8375-aea0014b2cea service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:26:05.796386 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-81af8a84-5964-4621-8375-aea0014b2cea service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:26:05.813708 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81af8a84-5964-4621-8375-aea0014b2cea service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:26:05.813708 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81af8a84-5964-4621-8375-aea0014b2cea service placement] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:26:05.822169 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1321/2619] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:26:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:26:15.063626 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=15 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:26:15.817697 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=9 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:26:20.766008 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123dd77b-c3c2-4fa8-8d2a-0f8d57819c46 service neutron] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:26:20.796852 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123dd77b-c3c2-4fa8-8d2a-0f8d57819c46 service neutron] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:26:20.803357 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce350a73-2047-420c-b8a6-dca00bbed2bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b07bf0faca47b28b50990d1720853f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:26:20.807931 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce350a73-2047-420c-b8a6-dca00bbed2bb service nova] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:26:20.807931 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce350a73-2047-420c-b8a6-dca00bbed2bb service nova] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:26:20.807931 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce350a73-2047-420c-b8a6-dca00bbed2bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:26:20.812810 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce350a73-2047-420c-b8a6-dca00bbed2bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:26:20.814423 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce350a73-2047-420c-b8a6-dca00bbed2bb service nova] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:26:20.823448 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1299/2620] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:26:20 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:26:30.819327 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=13 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:28:20.770609 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81af8a84-5964-4621-8375-aea0014b2cea service placement] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:28:20.779972 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81af8a84-5964-4621-8375-aea0014b2cea service placement] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:28:20.783491 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580d8661-3a63-4022-887a-78f868eb5853 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b123c412ac714b06929a4bd86eca71e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:28:20.784079 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580d8661-3a63-4022-887a-78f868eb5853 service neutron] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:28:20.784353 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580d8661-3a63-4022-887a-78f868eb5853 service neutron] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:28:20.784621 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580d8661-3a63-4022-887a-78f868eb5853 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:28:20.853236 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580d8661-3a63-4022-887a-78f868eb5853 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:28:20.854701 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580d8661-3a63-4022-887a-78f868eb5853 service neutron] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:28:20.873868 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1322/2621] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:28:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:28:24.369450 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce350a73-2047-420c-b8a6-dca00bbed2bb service nova] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:28:24.380927 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce350a73-2047-420c-b8a6-dca00bbed2bb service nova] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:28:24.386023 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d09669d-1bbe-4e2f-a6a5-200869b55469 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '922ef7811aed450baff7f936011b0af1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:28:24.389324 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d09669d-1bbe-4e2f-a6a5-200869b55469 service placement] REQUEST_METHOD: `GET` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:28:24.389324 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d09669d-1bbe-4e2f-a6a5-200869b55469 service placement] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:28:24.389324 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d09669d-1bbe-4e2f-a6a5-200869b55469 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:28:24.389324 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-2d09669d-1bbe-4e2f-a6a5-200869b55469 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:28:24.402524 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d09669d-1bbe-4e2f-a6a5-200869b55469 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:28:24.402524 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d09669d-1bbe-4e2f-a6a5-200869b55469 service placement] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:28:24.413218 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1300/2622] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:28:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:28:28.258726 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580d8661-3a63-4022-887a-78f868eb5853 service neutron] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:28:28.272948 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580d8661-3a63-4022-887a-78f868eb5853 service neutron] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:28:28.278677 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c4a2a9-f40e-495c-9c74-158c61c6a470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:28:28.279514 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c4a2a9-f40e-495c-9c74-158c61c6a470 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:28:28.279994 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c4a2a9-f40e-495c-9c74-158c61c6a470 admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:28:28.280442 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c4a2a9-f40e-495c-9c74-158c61c6a470 admin admin] PATH_INFO: `/v3/users/fcc0e4c5bee64180884513f13757140c` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:28:28.281799 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c4a2a9-f40e-495c-9c74-158c61c6a470 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fcc0e4c5bee64180884513f13757140c)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:28:28.284180 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c4a2a9-f40e-495c-9c74-158c61c6a470 admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:28:28.446376 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-82c4a2a9-f40e-495c-9c74-158c61c6a470 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fcc0e4c5bee64180884513f13757140c'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:28:28.447032 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-82c4a2a9-f40e-495c-9c74-158c61c6a470 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fcc0e4c5bee64180884513f13757140c'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:28:28.488481 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-82c4a2a9-f40e-495c-9c74-158c61c6a470 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fcc0e4c5bee64180884513f13757140c'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:28:28.489597 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-82c4a2a9-f40e-495c-9c74-158c61c6a470 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fcc0e4c5bee64180884513f13757140c'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:28:28.496833 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-82c4a2a9-f40e-495c-9c74-158c61c6a470 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fcc0e4c5bee64180884513f13757140c'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:28:28.514161 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1323/2623] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:28:28 2022] DELETE /identity/v3/users/fcc0e4c5bee64180884513f13757140c => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:28:28.764181 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d09669d-1bbe-4e2f-a6a5-200869b55469 service placement] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:28:28.775248 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d09669d-1bbe-4e2f-a6a5-200869b55469 service placement] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:28:28.830327 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ec5c02-0f09-4f35-acc1-6be0d3283c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:28:28.830949 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ec5c02-0f09-4f35-acc1-6be0d3283c9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:28:28.831223 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ec5c02-0f09-4f35-acc1-6be0d3283c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:28:28.831494 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ec5c02-0f09-4f35-acc1-6be0d3283c9e admin admin] PATH_INFO: `/v3/projects/094ec37b955e4518b7abffd09afef4f0` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:28:28.832589 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ec5c02-0f09-4f35-acc1-6be0d3283c9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=094ec37b955e4518b7abffd09afef4f0)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:28:28.834510 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ec5c02-0f09-4f35-acc1-6be0d3283c9e admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:28:28.935041 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-11ec5c02-0f09-4f35-acc1-6be0d3283c9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '094ec37b955e4518b7abffd09afef4f0', 'request_id': 'req-11ec5c02-0f09-4f35-acc1-6be0d3283c9e'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:28:28.941675 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-11ec5c02-0f09-4f35-acc1-6be0d3283c9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '094ec37b955e4518b7abffd09afef4f0', 'request_id': 'req-11ec5c02-0f09-4f35-acc1-6be0d3283c9e'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:28:28.997204 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-11ec5c02-0f09-4f35-acc1-6be0d3283c9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '094ec37b955e4518b7abffd09afef4f0', 'request_id': 'req-11ec5c02-0f09-4f35-acc1-6be0d3283c9e'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:28:29.020554 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-11ec5c02-0f09-4f35-acc1-6be0d3283c9e admin admin] The token cache is being invalidate because project 094ec37b955e4518b7abffd09afef4f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:28:29.020853 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-11ec5c02-0f09-4f35-acc1-6be0d3283c9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:28:29.021489 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-11ec5c02-0f09-4f35-acc1-6be0d3283c9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:28:29.022987 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1301/2624] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:28:28 2022] DELETE /identity/v3/projects/094ec37b955e4518b7abffd09afef4f0 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 14 03:28:29.026984 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c4a2a9-f40e-495c-9c74-158c61c6a470 admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:28:29.028425 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-82c4a2a9-f40e-495c-9c74-158c61c6a470 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:28:29.041864 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c4a2a9-f40e-495c-9c74-158c61c6a470 admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:28:29.084451 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:28:29.095622 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481e4173-7b44-4434-9510-6a58ea4ed57c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:28:29.096240 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481e4173-7b44-4434-9510-6a58ea4ed57c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:28:29.096521 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481e4173-7b44-4434-9510-6a58ea4ed57c admin admin] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:28:29.096794 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481e4173-7b44-4434-9510-6a58ea4ed57c admin admin] PATH_INFO: `/v3/users/f58f655bc33b4577bfaa3b6aa00882ba` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:28:29.097937 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481e4173-7b44-4434-9510-6a58ea4ed57c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f58f655bc33b4577bfaa3b6aa00882ba)` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:28:29.100431 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481e4173-7b44-4434-9510-6a58ea4ed57c admin admin] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:28:29.319201 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-481e4173-7b44-4434-9510-6a58ea4ed57c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f58f655bc33b4577bfaa3b6aa00882ba'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:28:29.319850 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-481e4173-7b44-4434-9510-6a58ea4ed57c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f58f655bc33b4577bfaa3b6aa00882ba'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:28:29.380658 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-481e4173-7b44-4434-9510-6a58ea4ed57c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f58f655bc33b4577bfaa3b6aa00882ba'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:28:29.381281 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-481e4173-7b44-4434-9510-6a58ea4ed57c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f58f655bc33b4577bfaa3b6aa00882ba'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:28:29.388535 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.notifications [None req-481e4173-7b44-4434-9510-6a58ea4ed57c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f58f655bc33b4577bfaa3b6aa00882ba'} {{(pid=81517) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:28:29.406528 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1324/2625] 10.222.0.39 () {58 vars in 1224 bytes} [Fri Oct 14 03:28:29 2022] DELETE /identity/v3/users/f58f655bc33b4577bfaa3b6aa00882ba => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:28:29.633460 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ec5c02-0f09-4f35-acc1-6be0d3283c9e admin admin] Authenticating user token {{(pid=81518) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:28:29.635009 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.fernet_utils [None req-11ec5c02-0f09-4f35-acc1-6be0d3283c9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81518) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:28:29.650319 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ec5c02-0f09-4f35-acc1-6be0d3283c9e admin admin] Validating token access rules against request {{(pid=81518) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:28:29.705608 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5418fc36-544b-430a-b372-c5e9b5282b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316b39adbf9f4b0d9c13dc4c11a24e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e86df8ec1d224299a71e492e2b3fb74d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81518) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:28:29.706266 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5418fc36-544b-430a-b372-c5e9b5282b50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:28:29.706546 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5418fc36-544b-430a-b372-c5e9b5282b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:28:29.706814 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5418fc36-544b-430a-b372-c5e9b5282b50 admin admin] PATH_INFO: `/v3/projects/ce079564be954618a73a9c64c4f551d5` {{(pid=81518) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:28:29.708548 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5418fc36-544b-430a-b372-c5e9b5282b50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce079564be954618a73a9c64c4f551d5)` {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:28:29.710482 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5418fc36-544b-430a-b372-c5e9b5282b50 admin admin] RBAC: Authorization granted {{(pid=81518) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:28:29.717974 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:28:29.826562 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-5418fc36-544b-430a-b372-c5e9b5282b50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce079564be954618a73a9c64c4f551d5', 'request_id': 'req-5418fc36-544b-430a-b372-c5e9b5282b50'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:28:29.832764 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-5418fc36-544b-430a-b372-c5e9b5282b50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce079564be954618a73a9c64c4f551d5', 'request_id': 'req-5418fc36-544b-430a-b372-c5e9b5282b50'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:28:29.911989 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-5418fc36-544b-430a-b372-c5e9b5282b50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce079564be954618a73a9c64c4f551d5', 'request_id': 'req-5418fc36-544b-430a-b372-c5e9b5282b50'} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:28:29.922759 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-5418fc36-544b-430a-b372-c5e9b5282b50 admin admin] The token cache is being invalidate because project ce079564be954618a73a9c64c4f551d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81518) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 14 03:28:29.923047 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-5418fc36-544b-430a-b372-c5e9b5282b50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:28:29.923670 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG keystone.notifications [None req-5418fc36-544b-430a-b372-c5e9b5282b50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81518) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 14 03:28:29.925345 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: [pid: 81518|app: 0|req: 1302/2626] 10.222.0.39 () {58 vars in 1233 bytes} [Fri Oct 14 03:28:29 2022] DELETE /identity/v3/projects/ce079564be954618a73a9c64c4f551d5 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 14 03:28:39.403048 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=31,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:28:39.919497 dv-cinder-801522-13-iscsi devstack@keystone.service[81518]: DEBUG dbcounter [-] [81518] Writing DB stats keystone:SELECT=21,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81518) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 14 03:30:55.694222 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481e4173-7b44-4434-9510-6a58ea4ed57c admin admin] Authenticating user token {{(pid=81517) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 14 03:30:55.695612 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-481e4173-7b44-4434-9510-6a58ea4ed57c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:30:55.764229 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481e4173-7b44-4434-9510-6a58ea4ed57c admin admin] Validating token access rules against request {{(pid=81517) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 14 03:30:55.821630 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd7d419-c352-4091-82ff-9808b6626a5a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '922ef7811aed450baff7f936011b0af1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dac9f8cd285c44e889a621b8b7fe479e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81517) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 14 03:30:55.822835 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd7d419-c352-4091-82ff-9808b6626a5a service placement] REQUEST_METHOD: `GET` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 14 03:30:55.823116 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd7d419-c352-4091-82ff-9808b6626a5a service placement] SCRIPT_NAME: `/identity` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 14 03:30:55.823392 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd7d419-c352-4091-82ff-9808b6626a5a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81517) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 14 03:30:55.825111 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.fernet_utils [None req-4fd7d419-c352-4091-82ff-9808b6626a5a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81517) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 14 03:30:55.836074 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd7d419-c352-4091-82ff-9808b6626a5a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 14 03:30:55.837640 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd7d419-c352-4091-82ff-9808b6626a5a service placement] RBAC: Authorization granted {{(pid=81517) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 14 03:30:55.847522 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: [pid: 81517|app: 0|req: 1325/2627] 10.222.0.39 () {60 vars in 1365 bytes} [Fri Oct 14 03:30:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 14 03:31:05.843179 dv-cinder-801522-13-iscsi devstack@keystone.service[81517]: DEBUG dbcounter [-] [81517] Writing DB stats keystone:SELECT=58 {{(pid=81517) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}